Refactor gitlab modules (#51141)

* gitlab_group: refactor module

* gitlab_user: refactor module

* gitlab_group, gitlab_user; pylint

* gitlab_project: refactor module

* gitlab_group, gitlab_project, gitlab_user: Enchance modules

- Add generic loop to update object
- Enchance return messages
- PyLint

* gitlab_runner: refactor module

* gitlab_hooks: refactor module

* gitlab_deploy_key: refactor module

* gitlab_group: enchance module and documentation

- Enchange function arguments
- Add check_mode break
- Rewrite module documentation

* gitlab_hook: enchance module and documentation

- Rewrite documentation
- Enchance function parameters
- Rename functions

* gitlab_project: enchance module and documentation

- Rewrite documentation
- Enchance function parameters
- Add try/except on project creation

* gitlab_runner: enchance module and documentation

- Rewrite documentation
- Fix Copyright
- Enchance function arguments
- Add check_mode break
- Add missing function: deletion

* gitlab_user: enchance module and documentation

- Rewrite documentation
- Enchance function parameters
- Add check_mode break
- Add try/except on user creation

* gitlab_deploy_key, gitlab_group, gitlab_hooks, gitlab_project,
gitlab_runner, gitlab_user: Fix residual bugs

- Fix Copyright
- Fix result messages
- Add missing check_mode break

* gitlab_deploy_key, gitlab_group, gitlab_hooks, gitlab_project, gitlab_runner, gitlab_user: pylint

* gitlab_runner: Add substitution function for 'cmp' in python3

* unit-test: remove deprecated gitlab module tests

- gitlab_deploy_key
- gitlab_hooks
- gitlab_project

Actually, they can't be reused because of the modification of the way that the module communicate with the Gitlab instance. It doesn't make direct call to the API, now it use a python library that do the job. So using a pytest mocker to test the module won't work.

* gitlab_deploy_key, gitlab_group, gitlab_hooks, gitlab_project, gitlab_runner, gitlab_user: add copyright

* gitlab_deploy_key, gitlab_group, gitlab_hooks, gitlab_project, gitlab_runner, gitlab_user: Support old parameters format

* module_utils Gitlab: Edit copyright

* gitlab_deploy_key, gitlab_group, gitlab_hooks, gitlab_project,
gitlab_runner, gitlab_user: Unifying module inputs

- Rename verify_ssl into validate_certs to match standards
- Remove unused alias parameters
- Unify parameters type and requirement
- Reorder list order

* gitlab_deploy_key, gitlab_group, gitlab_hooks, gitlab_project, gitlab_runner, gitlab_user: Unifying module outputs

- Use standard output parameter "msg" instead of "return"
- Use snail_case for return values instead of camelCase

* validate-module: remove sanity ignore

* BOTMETA: remove gitlab_* test

- This tests need to be completely rewriten because of the refactoring
of these modules
- TodoList Community Wiki was updated

* gitlab_user: Fix group identifier

* gitlab_project: Fix when group was empty

* gitlab_deploy_key: edit return msg

* module_utils gitlab: fall back to user namespace is project not found

* gitlab modules: Add units tests

* unit test: gitlab module fake current user

* gitlab_user: fix access_level verification

* gitlab unit tests: use decoration instead of with statement

* unit tests: gitlab module skip python 2.6

* unit tests: gitlab module skip library import if python 2.6

* gitlab unit tests: use builtin unittest class

* gitlab unit tests: use custom test class

* unit test: gitlab module lint

* unit tests: move gitlab utils

* unit test: gitlab fix imports

* gitlab_module: edit requirement

python-gitlab library require python >= 2.7

* gitlab_module: add myself as author

* gitlab_modules: add python encoding tag

* gitlab_modules: keep consistency between variable name "validate_certs"

* gitlab_modules: enchance documentation

* gitlab_runner: fix syntax error in documentation

* gitlab_module: use basic_auth module_utils and add deprecation warning

* gitlab_module: documentation corrections

* gitlab_module: python lint

* gitlab_module: deprecate options and aliases for ansible 2.10

* gitlab_group: don't use 'local_action' is documentation example

* gitlab_module: correct return messages

* gitlab_module: use module_util 'missing_required_lib' when python library is missing

* gitlab_module: fix typo in function name.

* gitlab_modules: unify return msg on check_mode

* gitlab_modules: don't use deprecated options in examples

.github/BOTMETA.yml

@@ -1312,7 +1312,6 @@ files:
     maintainers: $team_nxos
     labels:
       - networking
-  test/units/modules/source_control/test_gitlab_: *gitlab
   test/sanity/pep8/legacy-files.txt:
     notified: mattclay
   hacking/report.py:

lib/ansible/module_utils/gitlab.py

@@ -1,8 +1,10 @@
 # -*- coding: utf-8 -*-
 
-# (c) 2018, Marcus Watkins <marwatk@marcuswatkins.net>
+# Copyright: (c) 2019, Guillaume Martinez (guillaume.lunik@gmail.com)
+# Copyright: (c) 2018, Marcus Watkins <marwatk@marcuswatkins.net>
 # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
 
+from __future__ import absolute_import
 import json
 
 from ansible.module_utils.urls import fetch_url
@@ -35,3 +37,25 @@ def request(module, api_url, project, path, access_token, private_token, rawdata
         return True, json.loads(content)
     else:
         return False, str(status) + ": " + content
+
+
+def findProject(gitlab_instance, identifier):
+    try:
+        project = gitlab_instance.projects.get(identifier)
+    except Exception as e:
+        current_user = gitlab_instance.user
+        try:
+            project = gitlab_instance.projects.get(current_user.username + '/' + identifier)
+        except Exception as e:
+            return None
+
+    return project
+
+
+def findGroup(gitlab_instance, identifier):
+    try:
+        project = gitlab_instance.groups.get(identifier)
+    except Exception as e:
+        return None
+
+    return project

lib/ansible/modules/source_control/gitlab_deploy_key.py

@@ -1,91 +1,95 @@
 #!/usr/bin/python
 # -*- coding: utf-8 -*-
 
-# (c) 2018, Marcus Watkins <marwatk@marcuswatkins.net>
+# Copyright: (c) 2019, Guillaume Martinez (guillaume.lunik@gmail.com)
+# Copyright: (c) 2018, Marcus Watkins <marwatk@marcuswatkins.net>
 # Based on code:
-# (c) 2013, Phillip Gentry <phillip@cx.com>
+# Copyright: (c) 2013, Phillip Gentry <phillip@cx.com>
 # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
 
 from __future__ import absolute_import, division, print_function
 __metaclass__ = type
 
-
 ANSIBLE_METADATA = {'metadata_version': '1.1',
                     'status': ['preview'],
                     'supported_by': 'community'}
 
-
 DOCUMENTATION = '''
----
 module: gitlab_deploy_key
 short_description: Manages GitLab project deploy keys.
 description:
      - Adds, updates and removes project deploy keys
 version_added: "2.6"
+author:
+  - Marcus Watkins (@marwatk)
+  - Guillaume Martinez (@Lunik)
+requirements:
+  - python >= 2.7
+  - python-gitlab python module
+extends_documentation_fragment:
+    - auth_basic
 options:
-  api_url:
+  api_token:
-    description:
-      - GitLab API url, e.g. https://gitlab.example.com/api
-    required: true
-  access_token:
     description:
-      - The oauth key provided by GitLab. One of access_token or private_token is required. See https://docs.gitlab.com/ee/api/oauth2.html
+      - Gitlab token for logging in.
-    required: false
+    version_added: "2.8"
-  private_token:
+    type: str
-    description:
+    aliases:
-      - Personal access token to use. One of private_token or access_token is required. See https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html
+      - private_token
-    required: false
+      - access_token
   project:
     description:
-      - Numeric project id or name of project in the form of group/name
+      - Id or Full path of project in the form of group/name
     required: true
+    type: str
   title:
     description:
       - Deploy key's title
     required: true
+    type: str
   key:
     description:
       - Deploy key
     required: true
+    type: str
   can_push:
     description:
       - Whether this key can push to the project
     type: bool
-    default: 'no'
+    default: no
   state:
     description:
       - When C(present) the deploy key added to the project if it doesn't exist.
       - When C(absent) it will be removed from the project if it exists
     required: true
     default: present
+    type: str
     choices: [ "present", "absent" ]
-author: "Marcus Watkins (@marwatk)"
 '''
 
 EXAMPLES = '''
-# Example adding a project deploy key
+- name: "Adding a project deploy key"
-- gitlab_deploy_key:
+  gitlab_deploy_key:
-    api_url: https://gitlab.example.com/api
+    api_url: https://gitlab.example.com/
-    access_token: "{{ access_token }}"
+    api_token: "{{ access_token }}"
     project: "my_group/my_project"
     title: "Jenkins CI"
     state: present
     key: "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAiPWx6WM4lhHNedGfBpPJNPpZ7yKu+dnn1SJejgt4596k6YjzGGphH2TUxwKzxcKDKKezwkpfnxPkSMkuEspGRt/aZZ9w..."
 
-# Update the above deploy key to add push access
+- name: "Update the above deploy key to add push access"
-- gitlab_deploy_key:
+  gitlab_deploy_key:
-    api_url: https://gitlab.example.com/api
+    api_url: https://gitlab.example.com/
-    access_token: "{{ access_token }}"
+    api_token: "{{ access_token }}"
     project: "my_group/my_project"
     title: "Jenkins CI"
     state: present
-    key: "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAiPWx6WM4lhHNedGfBpPJNPpZ7yKu+dnn1SJejgt4596k6YjzGGphH2TUxwKzxcKDKKezwkpfnxPkSMkuEspGRt/aZZ9w..."
     can_push: yes
 
-# Remove the previous deploy key from the project
+- name: "Remove the previous deploy key from the project"
-- gitlab_deploy_key:
+  gitlab_deploy_key:
-    api_url: https://gitlab.example.com/api
+    api_url: https://gitlab.example.com/
-    access_token: "{{ access_token }}"
+    api_token: "{{ access_token }}"
     project: "my_group/my_project"
     state: absent
     key: "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAiPWx6WM4lhHNedGfBpPJNPpZ7yKu+dnn1SJejgt4596k6YjzGGphH2TUxwKzxcKDKKezwkpfnxPkSMkuEspGRt/aZZ9w..."
@@ -110,124 +114,213 @@ error:
   type: str
   sample: "400: key is already in use"
 
-previous_version:
+deploy_key:
-    description: object describing the state prior to this task
+  description: API object
-    returned: changed
+  returned: always
   type: dict
 '''
 
+import os
+import re
+import traceback
 
-import json
+GITLAB_IMP_ERR = None
+try:
+    import gitlab
+    HAS_GITLAB_PACKAGE = True
+except Exception:
+    GITLAB_IMP_ERR = traceback.format_exc()
+    HAS_GITLAB_PACKAGE = False
 
-from ansible.module_utils.basic import AnsibleModule
+from ansible.module_utils.api import basic_auth_argument_spec
-from copy import deepcopy
+from ansible.module_utils.basic import AnsibleModule, missing_required_lib
-from ansible.module_utils.gitlab import request
+from ansible.module_utils._text import to_native
 
+from ansible.module_utils.gitlab import findProject
 
-def _list(module, api_url, project, access_token, private_token):
-    path = "/deploy_keys"
-    return request(module, api_url, project, path, access_token, private_token)
 
+class GitLabDeployKey(object):
+    def __init__(self, module, gitlab_instance):
+        self._module = module
+        self._gitlab = gitlab_instance
+        self.deployKeyObject = None
 
-def _find(module, api_url, project, key, access_token, private_token):
+    '''
-    success, data = _list(module, api_url, project, access_token, private_token)
+    @param project Project object
-    if success:
+    @param key_title Title of the key
-        for i in data:
+    @param key_key String of the key
-            if i["key"] == key:
+    @param key_can_push Option of the deployKey
-                return success, i
+    @param options Deploy key options
-        return success, None
+    '''
-    return success, data
+    def createOrUpdateDeployKey(self, project, key_title, key_key, options):
+        changed = False
 
+        # Because we have already call existsDeployKey in main()
+        if self.deployKeyObject is None:
+            deployKey = self.createDeployKey(project, {
+                'title': key_title,
+                'key': key_key,
+                'can_push': options['can_push']})
+            changed = True
+        else:
+            changed, deployKey = self.updateDeployKey(self.deployKeyObject, {
+                'can_push': options['can_push']})
+
+        self.deployKeyObject = deployKey
+        if changed:
+            if self._module.check_mode:
+                self._module.exit_json(changed=True, msg="Successfully created or updated the deploy key %s" % key_title)
+
+            try:
+                deployKey.save()
+            except Exception as e:
+                self._module.fail_json(msg="Failed to update deploy key: %s " % e)
+            return True
+        else:
+            return False
 
-def _publish(module, api_url, project, data, access_token, private_token):
+    '''
-    path = "/deploy_keys"
+    @param project Project Object
-    method = "POST"
+    @param arguments Attributs of the deployKey
-    if 'id' in data:
+    '''
-        path += "/%s" % str(data["id"])
+    def createDeployKey(self, project, arguments):
-        method = "PUT"
+        if self._module.check_mode:
-    data = deepcopy(data)
+            return True
-    data.pop('id', None)
-    return request(module, api_url, project, path, access_token, private_token, json.dumps(data, sort_keys=True), method)
 
+        try:
+            deployKey = project.keys.create(arguments)
+        except (gitlab.exceptions.GitlabCreateError) as e:
+            self._module.fail_json(msg="Failed to create deploy key: %s " % to_native(e))
 
-def _delete(module, api_url, project, key_id, access_token, private_token):
+        return deployKey
-    path = "/deploy_keys/%s" % str(key_id)
-    return request(module, api_url, project, path, access_token, private_token, method='DELETE')
 
+    '''
+    @param deployKey Deploy Key Object
+    @param arguments Attributs of the deployKey
+    '''
+    def updateDeployKey(self, deployKey, arguments):
+        changed = False
 
-def _are_equivalent(input, existing):
+        for arg_key, arg_value in arguments.items():
-    for key in ['title', 'key', 'can_push']:
+            if arguments[arg_key] is not None:
-        if key in input and key not in existing:
+                if getattr(deployKey, arg_key) != arguments[arg_key]:
-            return False
+                    setattr(deployKey, arg_key, arguments[arg_key])
-        if key not in input and key in existing:
+                    changed = True
-            return False
+
-        if not input[key] == existing[key]:
+        return (changed, deployKey)
+
+    '''
+    @param project Project object
+    @param key_title Title of the key
+    '''
+    def findDeployKey(self, project, key_title):
+        deployKeys = project.keys.list()
+        for deployKey in deployKeys:
+            if (deployKey.title == key_title):
+                return deployKey
+
+    '''
+    @param project Project object
+    @param key_title Title of the key
+    '''
+    def existsDeployKey(self, project, key_title):
+        # When project exists, object will be stored in self.projectObject.
+        deployKey = self.findDeployKey(project, key_title)
+        if deployKey:
+            self.deployKeyObject = deployKey
+            return True
         return False
+
+    def deleteDeployKey(self):
+        if self._module.check_mode:
             return True
 
+        return self.deployKeyObject.delete()
+
+
+def deprecation_warning(module):
+    deprecated_aliases = ['private_token', 'access_token']
+
+    module.deprecate("Aliases \'{aliases}\' are deprecated".format(aliases='\', \''.join(deprecated_aliases)), 2.10)
+
 
 def main():
+    argument_spec = basic_auth_argument_spec()
+    argument_spec.update(dict(
+        api_token=dict(type='str', no_log=True, aliases=["private_token", "access_token"]),
+        state=dict(type='str', default="present", choices=["absent", "present"]),
+        project=dict(type='str', required=True),
+        key=dict(type='str', required=True),
+        can_push=dict(type='bool', default=False),
+        title=dict(type='str', required=True)
+    ))
+
     module = AnsibleModule(
-        argument_spec=dict(
+        argument_spec=argument_spec,
-            api_url=dict(required=True),
-            access_token=dict(required=False, no_log=True),
-            private_token=dict(required=False, no_log=True),
-            project=dict(required=True),
-            key=dict(required=True),
-            state=dict(default='present', choices=['present', 'absent']),
-            can_push=dict(default='no', type='bool'),
-            title=dict(required=True),
-        ),
         mutually_exclusive=[
-            ['access_token', 'private_token']
+            ['api_username', 'api_token'],
+            ['api_password', 'api_token']
+        ],
+        required_together=[
+            ['api_username', 'api_password']
         ],
         required_one_of=[
-            ['access_token', 'private_token']
+            ['api_username', 'api_token']
         ],
         supports_check_mode=True,
     )
 
-    api_url = module.params['api_url']
+    deprecation_warning(module)
-    access_token = module.params['access_token']
+
-    private_token = module.params['private_token']
+    gitlab_url = re.sub('/api.*', '', module.params['api_url'])
-    project = module.params['project']
+    validate_certs = module.params['validate_certs']
+    gitlab_user = module.params['api_username']
+    gitlab_password = module.params['api_password']
+    gitlab_token = module.params['api_token']
+
     state = module.params['state']
+    project_identifier = module.params['project']
+    key_title = module.params['title']
+    key_keyfile = module.params['key']
+    key_can_push = module.params['can_push']
 
-    if not access_token and not private_token:
+    if not HAS_GITLAB_PACKAGE:
-        module.fail_json(msg="need either access_token or private_token")
+        module.fail_json(msg=missing_required_lib("python-gitlab"), exception=GITLAB_IMP_ERR)
 
-    input = {}
+    try:
+        gitlab_instance = gitlab.Gitlab(url=gitlab_url, ssl_verify=validate_certs, email=gitlab_user, password=gitlab_password,
+                                        private_token=gitlab_token, api_version=4)
+        gitlab_instance.auth()
+    except (gitlab.exceptions.GitlabAuthenticationError, gitlab.exceptions.GitlabGetError) as e:
+        module.fail_json(msg="Failed to connect to Gitlab server: %s" % to_native(e))
+    except (gitlab.exceptions.GitlabHttpError) as e:
+        module.fail_json(msg="Failed to connect to Gitlab server: %s. \
+            Gitlab remove Session API now that private tokens are removed from user API endpoints since version 10.2." % to_native(e))
 
-    for key in ['title', 'key', 'can_push']:
+    gitlab_deploy_key = GitLabDeployKey(module, gitlab_instance)
-        input[key] = module.params[key]
 
-    success, existing = _find(module, api_url, project, input['key'], access_token, private_token)
+    project = findProject(gitlab_instance, project_identifier)
 
-    if not success:
+    if project is None:
-        module.fail_json(msg="failed to list deploy keys", result=existing)
+        module.fail_json(msg="Failed to create deploy key: project %s doesn't exists" % project_identifier)
 
-    if existing:
+    deployKey_exists = gitlab_deploy_key.existsDeployKey(project, key_title)
-        input['id'] = existing['id']
 
-    changed = False
+    if state == 'absent':
-    success = True
+        if deployKey_exists:
-    response = None
+            gitlab_deploy_key.deleteDeployKey()
+            module.exit_json(changed=True, msg="Successfully deleted deploy key %s" % key_title)
+        else:
+            module.exit_json(changed=False, msg="Deploy key deleted or does not exists")
 
     if state == 'present':
-        if not existing or not _are_equivalent(existing, input):
+        if gitlab_deploy_key.createOrUpdateDeployKey(project, key_title, key_keyfile, {'can_push': key_can_push}):
-            if not module.check_mode:
-                success, response = _publish(module, api_url, project, input, access_token, private_token)
-            changed = True
-    else:
-        if existing:
-            if not module.check_mode:
-                success, response = _delete(module, api_url, project, existing['id'], access_token, private_token)
-            changed = True
 
-    if success:
+            module.exit_json(changed=True, msg="Successfully created or updated the deploy key %s" % key_title,
-        module.exit_json(changed=changed, msg='Success', result=response, previous_version=existing)
+                             deploy_key=gitlab_deploy_key.deployKeyObject._attrs)
         else:
-        module.fail_json(msg='Failure', error=response)
+            module.exit_json(changed=False, msg="No need to update the deploy key %s" % key_title,
+                             deploy_key=gitlab_deploy_key.deployKeyObject._attrs)
 
 
 if __name__ == '__main__':

lib/ansible/modules/source_control/gitlab_group.py

@@ -1,5 +1,7 @@
 #!/usr/bin/python
+# -*- coding: utf-8 -*-
 
+# Copyright: (c) 2019, Guillaume Martinez (guillaume.lunik@gmail.com)
 # Copyright: (c) 2015, Werner Dijkerman (ikben@werner-dijkerman.nl)
 # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
 
@@ -17,201 +19,362 @@ short_description: Creates/updates/deletes Gitlab Groups
 description:
   - When the group does not exist in Gitlab, it will be created.
   - When the group does exist and state=absent, the group will be deleted.
-   - As of Ansible version 2.7, this module make use of a different python module and thus some arguments are deprecated.
 version_added: "2.1"
-author: "Werner Dijkerman (@dj-wasabi)"
+author:
+  - Werner Dijkerman (@dj-wasabi)
+  - Guillaume Martinez (@Lunik)
 requirements:
+  - python >= 2.7
   - python-gitlab python module
+extends_documentation_fragment:
+    - auth_basic
 options:
   server_url:
     description:
-            - Url of Gitlab server, with protocol (http or https).
+      - The URL of the Gitlab server, with protocol (i.e. http or https).
     required: true
-    validate_certs:
+    type: str
-        description:
-            - When using https if SSL certificate needs to be verified.
-        type: bool
-        default: 'yes'
-        aliases:
-            - verify_ssl
   login_user:
     description:
       - Gitlab user name.
+    type: str
   login_password:
     description:
       - Gitlab password for login_user
-    login_token:
+    type: str
+  api_token:
     description:
       - Gitlab token for logging in.
+    type: str
+    aliases:
+      - login_token
   name:
     description:
       - Name of the group you want to create.
     required: true
+    type: str
   path:
     description:
       - The path of the group you want to create, this will be server_url/group_path
       - If not supplied, the group_name will be used.
+    type: str
   description:
     description:
       - A description for the group.
     version_added: "2.7"
+    type: str
   state:
     description:
       - create or delete group.
       - Possible values are present and absent.
-        default: "present"
+    default: present
+    type: str
     choices: ["present", "absent"]
+  parent:
+    description:
+      - Allow to create subgroups
+      - Id or Full path of parent group in the form of group/name
+    version_added: "2.8"
+    type: str
+  visibility:
+    description:
+      - Default visibility of the group
+    version_added: "2.8"
+    choices: ["private", "internal", "public"]
+    default: private
+    type: str
 '''
 
 EXAMPLES = '''
 - name: "Delete Gitlab Group"
-  local_action:
   gitlab_group:
-        server_url: http://gitlab.dj-wasabi.local
+    api_url: https://gitlab.example.com/
+    api_token: "{{ access_token }}"
     validate_certs: False
-        login_token: WnUzDsxjy8230-Dy_k
     name: my_first_group
     state: absent
 
 - name: "Create Gitlab Group"
-  local_action:
   gitlab_group:
-        server_url: https://gitlab.dj-wasabi.local"
+    api_url: https://gitlab.example.com/
     validate_certs: True
-        login_user: dj-wasabi
+    api_usersername: dj-wasabi
-        login_password: "MySecretPassword"
+    api_password: "MySecretPassword"
     name: my_first_group
     path: my_first_group
     state: present
+
+# The group will by created at https://gitlab.dj-wasabi.local/super_parent/parent/my_first_group
+- name: "Create Gitlab SubGroup"
+  gitlab_group:
+    api_url: https://gitlab.example.com/
+    validate_certs: True
+    api_usersername: dj-wasabi
+    api_password: "MySecretPassword"
+    name: my_first_group
+    path: my_first_group
+    state: present
+    parent_path: "super_parent/parent"
+'''
+
+RETURN = '''
+msg:
+  description: Success or failure message
+  returned: always
+  type: str
+  sample: "Success"
+
+result:
+  description: json parsed response from the server
+  returned: always
+  type: dict
+
+error:
+  description: the error message returned by the Gitlab API
+  returned: failed
+  type: str
+  sample: "400: path is already in use"
+
+group:
+  description: API object
+  returned: always
+  type: dict
 '''
 
-RETURN = '''# '''
+import os
+import traceback
 
+GITLAB_IMP_ERR = None
 try:
     import gitlab
     HAS_GITLAB_PACKAGE = True
 except Exception:
+    GITLAB_IMP_ERR = traceback.format_exc()
     HAS_GITLAB_PACKAGE = False
 
-from ansible.module_utils.basic import AnsibleModule
+from ansible.module_utils.api import basic_auth_argument_spec
+from ansible.module_utils.basic import AnsibleModule, missing_required_lib
 from ansible.module_utils._text import to_native
 
+from ansible.module_utils.gitlab import findGroup
+
 
 class GitLabGroup(object):
-    def __init__(self, module, git):
+    def __init__(self, module, gitlab_instance):
         self._module = module
-        self._gitlab = git
+        self._gitlab = gitlab_instance
         self.groupObject = None
 
-    def createOrUpdateGroup(self, name, path, description):
+    '''
+    @param group Group object
+    '''
+    def getGroupId(self, group):
+        if group is not None:
+            return group.id
+        return None
+
+    '''
+    @param name Name of the group
+    @param parent Parent group full path
+    @param options Group options
+    '''
+    def createOrUpdateGroup(self, name, parent, options):
         changed = False
+
+        # Because we have already call userExists in main()
         if self.groupObject is None:
-            group = self._gitlab.groups.create({'name': name, 'path': path})
+            parent_id = self.getGroupId(parent)
-            changed = True
-        else:
-            group = self.groupObject
 
-        if description is not None:
+            group = self.createGroup({
-            if group.description != description:
+                'name': name,
-                group.description = description
+                'path': options['path'],
+                'parent_id': parent_id})
             changed = True
+        else:
+            changed, group = self.updateGroup(self.groupObject, {
+                'name': name,
+                'description': options['description'],
+                'visibility': options['visibility']})
 
+        self.groupObject = group
         if changed:
             if self._module.check_mode:
-                self._module.exit_json(changed=True, result="Group should have updated.")
+                self._module.exit_json(changed=True, msg="Successfully created or updated the group %s" % name)
+
             try:
                 group.save()
             except Exception as e:
-                self._module.fail_json(msg="Failed to create or update a group: %s " % e)
+                self._module.fail_json(msg="Failed to update group: %s " % e)
             return True
         else:
             return False
 
+    '''
+    @param arguments Attributs of the group
+    '''
+    def createGroup(self, arguments):
+        if self._module.check_mode:
+            return True
+
+        try:
+            group = self._gitlab.groups.create(arguments)
+        except (gitlab.exceptions.GitlabCreateError) as e:
+            self._module.fail_json(msg="Failed to create group: %s " % to_native(e))
+
+        return group
+
+    '''
+    @param group Group Object
+    @param arguments Attributs of the group
+    '''
+    def updateGroup(self, group, arguments):
+        changed = False
+
+        for arg_key, arg_value in arguments.items():
+            if arguments[arg_key] is not None:
+                if getattr(group, arg_key) != arguments[arg_key]:
+                    setattr(group, arg_key, arguments[arg_key])
+                    changed = True
+
+        return (changed, group)
+
     def deleteGroup(self):
         group = self.groupObject
+
         if len(group.projects.list()) >= 1:
             self._module.fail_json(
                 msg="There are still projects in this group. These needs to be moved or deleted before this group can be removed.")
         else:
             if self._module.check_mode:
-                self._module.exit_json(changed=True)
+                return True
+
             try:
                 group.delete()
             except Exception as e:
-                self._module.fail_json(msg="Failed to delete a group: %s " % e)
+                self._module.fail_json(msg="Failed to delete group: %s " % to_native(e))
-            return True
 
-    def existsGroup(self, name):
+    '''
-        """When group/user exists, object will be stored in self.groupObject."""
+    @param name Name of the groupe
-        groups = self._gitlab.groups.list(search=name)
+    @param full_path Complete path of the Group including parent group path. <parent_path>/<group_path>
-        if len(groups) == 1:
+    '''
-            self.groupObject = groups[0]
+    def existsGroup(self, project_identifier):
+        # When group/user exists, object will be stored in self.groupObject.
+        group = findGroup(self._gitlab, project_identifier)
+        if group:
+            self.groupObject = group
             return True
+        return False
+
+
+def deprecation_warning(module):
+    deprecated_aliases = ['login_token']
+
+    module.deprecate("Aliases \'{aliases}\' are deprecated".format(aliases='\', \''.join(deprecated_aliases)), 2.10)
 
 
 def main():
+    argument_spec = basic_auth_argument_spec()
+    argument_spec.update(dict(
+        server_url=dict(type='str', required=True, removed_in_version=2.10),
+        login_user=dict(type='str', no_log=True, removed_in_version=2.10),
+        login_password=dict(type='str', no_log=True, removed_in_version=2.10),
+        api_token=dict(type='str', no_log=True, aliases=["login_token"]),
+        name=dict(type='str', required=True),
+        path=dict(type='str'),
+        description=dict(type='str'),
+        state=dict(type='str', default="present", choices=["absent", "present"]),
+        parent=dict(type='str'),
+        visibility=dict(type='str', default="private", choices=["internal", "private", "public"]),
+    ))
+
     module = AnsibleModule(
-        argument_spec=dict(
+        argument_spec=argument_spec,
-            server_url=dict(required=True, type='str'),
-            validate_certs=dict(required=False, default=True, type='bool', aliases=['verify_ssl']),
-            login_user=dict(required=False, no_log=True, type='str'),
-            login_password=dict(required=False, no_log=True, type='str'),
-            login_token=dict(required=False, no_log=True, type='str'),
-            name=dict(required=True, type='str'),
-            path=dict(required=False, type='str'),
-            description=dict(required=False, type='str'),
-            state=dict(default="present", choices=["present", "absent"]),
-        ),
         mutually_exclusive=[
+            ['api_url', 'server_url'],
+            ['api_username', 'login_user'],
+            ['api_password', 'login_password'],
+            ['api_username', 'api_token'],
+            ['api_password', 'api_token'],
             ['login_user', 'login_token'],
             ['login_password', 'login_token']
         ],
         required_together=[
-            ['login_user', 'login_password']
+            ['api_username', 'api_password'],
+            ['login_user', 'login_password'],
         ],
         required_one_of=[
-            ['login_user', 'login_token']
+            ['api_username', 'api_token', 'login_user', 'login_token']
         ],
-        supports_check_mode=True
+        supports_check_mode=True,
     )
 
-    if not HAS_GITLAB_PACKAGE:
+    deprecation_warning(module)
-        module.fail_json(msg="Missing required gitlab module (check docs or install with: pip install python-gitlab")
 
     server_url = module.params['server_url']
-    validate_certs = module.params['validate_certs']
     login_user = module.params['login_user']
     login_password = module.params['login_password']
-    login_token = module.params['login_token']
+
+    api_url = module.params['api_url']
+    validate_certs = module.params['validate_certs']
+    api_user = module.params['api_username']
+    api_password = module.params['api_password']
+
+    gitlab_url = server_url if api_url is None else api_url
+    gitlab_user = login_user if api_user is None else api_user
+    gitlab_password = login_password if api_password is None else api_password
+    gitlab_token = module.params['api_token']
+
     group_name = module.params['name']
     group_path = module.params['path']
     description = module.params['description']
     state = module.params['state']
+    parent_identifier = module.params['parent']
+    group_visibility = module.params['visibility']
+
+    if not HAS_GITLAB_PACKAGE:
+        module.fail_json(msg=missing_required_lib("python-gitlab"), exception=GITLAB_IMP_ERR)
 
     try:
-        git = gitlab.Gitlab(url=server_url, ssl_verify=validate_certs, email=login_user, password=login_password,
+        gitlab_instance = gitlab.Gitlab(url=gitlab_url, ssl_verify=validate_certs, email=gitlab_user, password=gitlab_password,
-                            private_token=login_token, api_version=4)
+                                        private_token=gitlab_token, api_version=4)
-        git.auth()
+        gitlab_instance.auth()
     except (gitlab.exceptions.GitlabAuthenticationError, gitlab.exceptions.GitlabGetError) as e:
-        module.fail_json(msg='Failed to connect to Gitlab server: %s' % to_native(e))
+        module.fail_json(msg="Failed to connect to Gitlab server: %s" % to_native(e))
+    except (gitlab.exceptions.GitlabHttpError) as e:
+        module.fail_json(msg="Failed to connect to Gitlab server: %s. \
+            Gitlab remove Session API now that private tokens are removed from user API endpoints since version 10.2" % to_native(e))
 
+    # Define default group_path based on group_name
     if group_path is None:
         group_path = group_name.replace(" ", "_")
 
-    group = GitLabGroup(module, git)
+    gitlab_group = GitLabGroup(module, gitlab_instance)
-    group_exists = group.existsGroup(group_name)
+
+    parent_group = None
+    if parent_identifier:
+        parent_group = findGroup(gitlab_instance, parent_identifier)
+        if not parent_group:
+            module.fail_json(msg="Failed create Gitlab group: Parent group doesn't exists")
 
-    if group_exists and state == "absent":
+        group_exists = gitlab_group.existsGroup(parent_group.full_path + '/' + group_path)
-        if group.deleteGroup():
-            module.exit_json(changed=True, result="Successfully deleted group %s" % group_name)
     else:
-        if state == "absent":
+        group_exists = gitlab_group.existsGroup(group_path)
-            module.exit_json(changed=False, result="Group deleted or does not exists")
+
+    if state == 'absent':
+        if group_exists:
+            gitlab_group.deleteGroup()
+            module.exit_json(changed=True, msg="Successfully deleted group %s" % group_name)
         else:
-            if group.createOrUpdateGroup(name=group_name, path=group_path, description=description):
+            module.exit_json(changed=False, msg="Group deleted or does not exists")
-                module.exit_json(changed=True, result="Successfully created or updated the group %s" % group_name)
+
+    if state == 'present':
+        if gitlab_group.createOrUpdateGroup(group_name, parent_group, {
+                                            "path": group_path,
+                                            "description": description,
+                                            "visibility": group_visibility}):
+            module.exit_json(changed=True, msg="Successfully created or updated the group %s" % group_name, group=gitlab_group.groupObject._attrs)
         else:
-                module.exit_json(changed=False, result="No need to update the group %s" % group_name)
+            module.exit_json(changed=False, msg="No need to update the group %s" % group_name, group=gitlab_group.groupObject._attrs)
 
 
 if __name__ == '__main__':

lib/ansible/modules/source_control/gitlab_hooks.py

@@ -1,20 +1,19 @@
 #!/usr/bin/python
 # -*- coding: utf-8 -*-
 
-# (c) 2018, Marcus Watkins <marwatk@marcuswatkins.net>
+# Copyright: (c) 2019, Guillaume Martinez (guillaume.lunik@gmail.com)
+# Copyright: (c) 2018, Marcus Watkins <marwatk@marcuswatkins.net>
 # Based on code:
-# (c) 2013, Phillip Gentry <phillip@cx.com>
+# Copyright: (c) 2013, Phillip Gentry <phillip@cx.com>
 # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
 
 from __future__ import absolute_import, division, print_function
 __metaclass__ = type
 
-
 ANSIBLE_METADATA = {'metadata_version': '1.1',
                     'status': ['preview'],
                     'supported_by': 'community'}
 
-
 DOCUMENTATION = '''
 ---
 module: gitlab_hooks
@@ -22,103 +21,99 @@ short_description: Manages GitLab project hooks.
 description:
      - Adds, updates and removes project hooks
 version_added: "2.6"
+author:
+  - Marcus Watkins (@marwatk)
+  - Guillaume Martinez (@Lunik)
+requirements:
+  - python >= 2.7
+  - python-gitlab python module
+extends_documentation_fragment:
+    - auth_basic
 options:
-  api_url:
+  api_token:
-    description:
-      - GitLab API url, e.g. https://gitlab.example.com/api
-    required: true
-  access_token:
     description:
-      - The oauth key provided by GitLab. One of access_token or private_token is required. See https://docs.gitlab.com/ee/api/oauth2.html
+      - Gitlab token for logging in.
-    required: false
+    version_added: "2.8"
-  private_token:
+    type: str
-    description:
+    aliases:
-      - Personal access token to use. One of private_token or access_token is required. See https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html
+      - private_token
-    required: false
+      - access_token
   project:
     description:
-      - Numeric project id or name of project in the form of group/name
+      - Id or Full path of the project in the form of group/name
     required: true
+    type: str
   hook_url:
     description:
       - The url that you want GitLab to post to, this is used as the primary key for updates and deletion.
     required: true
+    type: str
   state:
     description:
       - When C(present) the hook will be updated to match the input or created if it doesn't exist. When C(absent) it will be deleted if it exists.
     required: true
     default: present
+    type: str
     choices: [ "present", "absent" ]
   push_events:
     description:
       - Trigger hook on push events
     type: bool
-    default: 'yes'
+    default: yes
   issues_events:
     description:
       - Trigger hook on issues events
     type: bool
-    default: 'no'
+    default: no
   merge_requests_events:
     description:
       - Trigger hook on merge requests events
     type: bool
-    default: 'no'
+    default: no
   tag_push_events:
     description:
       - Trigger hook on tag push events
     type: bool
-    default: 'no'
+    default: no
   note_events:
     description:
       - Trigger hook on note events
     type: bool
-    default: 'no'
+    default: no
   job_events:
     description:
       - Trigger hook on job events
     type: bool
-    default: 'no'
+    default: no
   pipeline_events:
     description:
       - Trigger hook on pipeline events
     type: bool
-    default: 'no'
+    default: no
   wiki_page_events:
     description:
       - Trigger hook on wiki events
     type: bool
-    default: 'no'
+    default: no
   enable_ssl_verification:
     description:
       - Whether GitLab will do SSL verification when triggering the hook
     type: bool
-    default: 'no'
+    default: no
   token:
     description:
       - Secret token to validate hook messages at the receiver.
       - If this is present it will always result in a change as it cannot be retrieved from GitLab.
       - Will show up in the X-Gitlab-Token HTTP request header
     required: false
-author: "Marcus Watkins (@marwatk)"
+    type: str
 '''
 
 EXAMPLES = '''
-# Example creating a new project hook
+- name: "Adding a project hook"
-- gitlab_hooks:
+  gitlab_hooks:
-    api_url: https://gitlab.example.com/api
+    api_url: https://gitlab.example.com/
-    access_token: "{{ access_token }}"
+    api_token: "{{ access_token }}"
-    project: "my_group/my_project"
-    hook_url: "https://my-ci-server.example.com/gitlab-hook"
-    state: present
-    push_events: yes
-    enable_ssl_verification: no
-    token: "my-super-secret-token-that-my-ci-server-will-check"
-
-# Update the above hook to add tag pushes
-- gitlab_hooks:
-    api_url: https://gitlab.example.com/api
-    access_token: "{{ access_token }}"
     project: "my_group/my_project"
     hook_url: "https://my-ci-server.example.com/gitlab-hook"
     state: present
@@ -127,18 +122,18 @@ EXAMPLES = '''
     enable_ssl_verification: no
     token: "my-super-secret-token-that-my-ci-server-will-check"
 
-# Delete the previous hook
+- name: "Delete the previous hook"
-- gitlab_hooks:
+  gitlab_hooks:
-    api_url: https://gitlab.example.com/api
+    api_url: https://gitlab.example.com/
-    access_token: "{{ access_token }}"
+    api_token: "{{ access_token }}"
     project: "my_group/my_project"
     hook_url: "https://my-ci-server.example.com/gitlab-hook"
     state: absent
 
-# Delete a hook by numeric project id
+- name: "Delete a hook by numeric project id"
-- gitlab_hooks:
+  gitlab_hooks:
-    api_url: https://gitlab.example.com/api
+    api_url: https://gitlab.example.com/
-    access_token: "{{ access_token }}"
+    api_token: "{{ access_token }}"
     project: 10
     hook_url: "https://my-ci-server.example.com/gitlab-hook"
     state: absent
@@ -160,141 +155,252 @@ error:
   description: the error message returned by the Gitlab API
   returned: failed
   type: str
-    sample: "400: key is already in use"
+  sample: "400: path is already in use"
 
-previous_version:
+hook:
-    description: object describing the state prior to this task
+  description: API object
-    returned: changed
+  returned: always
   type: dict
 '''
 
+import os
+import re
+import traceback
 
-import json
+GITLAB_IMP_ERR = None
+try:
+    import gitlab
+    HAS_GITLAB_PACKAGE = True
+except Exception:
+    GITLAB_IMP_ERR = traceback.format_exc()
+    HAS_GITLAB_PACKAGE = False
 
-from ansible.module_utils.basic import AnsibleModule
+from ansible.module_utils.api import basic_auth_argument_spec
-from copy import deepcopy
+from ansible.module_utils.basic import AnsibleModule, missing_required_lib
+from ansible.module_utils._text import to_native
 
-from ansible.module_utils.gitlab import request
+from ansible.module_utils.gitlab import findProject
 
 
-def _list(module, api_url, project, access_token, private_token):
+class GitLabHook(object):
-    path = "/hooks"
+    def __init__(self, module, gitlab_instance):
-    return request(module, api_url, project, path, access_token, private_token)
+        self._module = module
+        self._gitlab = gitlab_instance
+        self.hookObject = None
+
+    '''
+    @param prokect Project Object
+    @param hook_url Url to call on event
+    @param description Description of the group
+    @param parent Parent group full path
+    '''
+    def createOrUpdateHook(self, project, hook_url, options):
+        changed = False
+
+        # Because we have already call userExists in main()
+        if self.hookObject is None:
+            hook = self.createHook(project, {
+                'url': hook_url,
+                'push_events': options['push_events'],
+                'issues_events': options['issues_events'],
+                'merge_requests_events': options['merge_requests_events'],
+                'tag_push_events': options['tag_push_events'],
+                'note_events': options['note_events'],
+                'job_events': options['job_events'],
+                'pipeline_events': options['pipeline_events'],
+                'wiki_page_events': options['wiki_page_events'],
+                'enable_ssl_verification': options['enable_ssl_verification'],
+                'token': options['token']})
+            changed = True
+        else:
+            changed, hook = self.updateHook(self.hookObject, {
+                'push_events': options['push_events'],
+                'issues_events': options['issues_events'],
+                'merge_requests_events': options['merge_requests_events'],
+                'tag_push_events': options['tag_push_events'],
+                'note_events': options['note_events'],
+                'job_events': options['job_events'],
+                'pipeline_events': options['pipeline_events'],
+                'wiki_page_events': options['wiki_page_events'],
+                'enable_ssl_verification': options['enable_ssl_verification'],
+                'token': options['token']})
+
+        self.hookObject = hook
+        if changed:
+            if self._module.check_mode:
+                self._module.exit_json(changed=True, msg="Successfully created or updated the hook %s" % hook_url)
+
+            try:
+                hook.save()
+            except Exception as e:
+                self._module.fail_json(msg="Failed to update hook: %s " % e)
+            return True
+        else:
+            return False
 
+    '''
+    @param project Project Object
+    @param arguments Attributs of the hook
+    '''
+    def createHook(self, project, arguments):
+        if self._module.check_mode:
+            return True
 
-def _find(module, api_url, project, hook_url, access_token, private_token):
+        hook = project.hooks.create(arguments)
-    success, data = _list(module, api_url, project, access_token, private_token)
-    if success:
-        for i in data:
-            if i["url"] == hook_url:
-                return success, i
-        return success, None
-    return success, data
 
+        return hook
 
-def _publish(module, api_url, project, data, access_token, private_token):
+    '''
-    path = "/hooks"
+    @param hook Hook Object
-    method = "POST"
+    @param arguments Attributs of the hook
-    if 'id' in data:
+    '''
-        path += "/%s" % str(data["id"])
+    def updateHook(self, hook, arguments):
-        method = "PUT"
+        changed = False
-    data = deepcopy(data)
-    data.pop('id', None)
-    return request(module, api_url, project, path, access_token, private_token, json.dumps(data, sort_keys=True), method)
 
+        for arg_key, arg_value in arguments.items():
+            if arguments[arg_key] is not None:
+                if getattr(hook, arg_key) != arguments[arg_key]:
+                    setattr(hook, arg_key, arguments[arg_key])
+                    changed = True
 
-def _delete(module, api_url, project, hook_id, access_token, private_token):
+        return (changed, hook)
-    path = "/hooks/%s" % str(hook_id)
-    return request(module, api_url, project, path, access_token, private_token, method='DELETE')
 
+    '''
+    @param project Project object
+    @param hook_url Url to call on event
+    '''
+    def findHook(self, project, hook_url):
+        hooks = project.hooks.list()
+        for hook in hooks:
+            if (hook.url == hook_url):
+                return hook
 
-def _are_equivalent(input, existing):
+    '''
-    for key in [
+    @param project Project object
-            'url', 'push_events', 'issues_events', 'merge_requests_events',
+    @param hook_url Url to call on event
-            'tag_push_events', 'note_events', 'job_events', 'pipeline_events', 'wiki_page_events',
+    '''
-            'enable_ssl_verification']:
+    def existsHooks(self, project, hook_url):
-        if key in input and key not in existing:
+        # When project exists, object will be stored in self.projectObject.
-            return False
+        hook = self.findHook(project, hook_url)
-        if key not in input and key in existing:
+        if hook:
-            return False
+            self.hookObject = hook
-        if not input[key] == existing[key]:
+            return True
         return False
+
+    def deleteHook(self):
+        if self._module.check_mode:
             return True
 
+        return self.hookObject.delete()
+
+
+def deprecation_warning(module):
+    deprecated_aliases = ['private_token', 'access_token']
+
+    module.deprecate("Aliases \'{aliases}\' are deprecated".format(aliases='\', \''.join(deprecated_aliases)), 2.10)
+
 
 def main():
+    argument_spec = basic_auth_argument_spec()
+    argument_spec.update(dict(
+        api_token=dict(type='str', no_log=True, aliases=["private_token", "access_token"]),
+        state=dict(type='str', default="present", choices=["absent", "present"]),
+        project=dict(type='str', required=True),
+        hook_url=dict(type='str', required=True),
+        push_events=dict(type='bool', default=True),
+        issues_events=dict(type='bool', default=False),
+        merge_requests_events=dict(type='bool', default=False),
+        tag_push_events=dict(type='bool', default=False),
+        note_events=dict(type='bool', default=False),
+        job_events=dict(type='bool', default=False),
+        pipeline_events=dict(type='bool', default=False),
+        wiki_page_events=dict(type='bool', default=False),
+        enable_ssl_verification=dict(type='bool', default=False),
+        token=dict(type='str', no_log=True),
+    ))
+
     module = AnsibleModule(
-        argument_spec=dict(
+        argument_spec=argument_spec,
-            api_url=dict(required=True),
-            access_token=dict(required=False, no_log=True),
-            private_token=dict(required=False, no_log=True),
-            project=dict(required=True),
-            hook_url=dict(required=True),
-            state=dict(default='present', choices=['present', 'absent']),
-            push_events=dict(default='yes', type='bool'),
-            issues_events=dict(default='no', type='bool'),
-            merge_requests_events=dict(default='no', type='bool'),
-            tag_push_events=dict(default='no', type='bool'),
-            note_events=dict(default='no', type='bool'),
-            job_events=dict(default='no', type='bool'),
-            pipeline_events=dict(default='no', type='bool'),
-            wiki_page_events=dict(default='no', type='bool'),
-            enable_ssl_verification=dict(default='no', type='bool'),
-            token=dict(required=False, no_log=True),
-        ),
         mutually_exclusive=[
-            ['access_token', 'private_token']
+            ['api_username', 'api_token'],
+            ['api_password', 'api_token']
+        ],
+        required_together=[
+            ['api_username', 'api_password']
         ],
         required_one_of=[
-            ['access_token', 'private_token']
+            ['api_username', 'api_token']
         ],
         supports_check_mode=True,
     )
 
-    api_url = module.params['api_url']
+    deprecation_warning(module)
-    access_token = module.params['access_token']
-    private_token = module.params['private_token']
-    project = module.params['project']
-    state = module.params['state']
-
-    if not access_token and not private_token:
-        module.fail_json(msg="need either access_token or private_token")
-
-    input = {'url': module.params['hook_url']}
-
-    for key in [
-            'push_events', 'issues_events', 'merge_requests_events',
-            'tag_push_events', 'note_events', 'job_events', 'pipeline_events', 'wiki_page_events',
-            'enable_ssl_verification', 'token']:
-        input[key] = module.params[key]
-
-    success, existing = _find(module, api_url, project, input['url'], access_token, private_token)
 
-    if not success:
+    gitlab_url = re.sub('/api.*', '', module.params['api_url'])
-        module.fail_json(msg="failed to list hooks", result=existing)
+    validate_certs = module.params['validate_certs']
+    gitlab_user = module.params['api_username']
+    gitlab_password = module.params['api_password']
+    gitlab_token = module.params['api_token']
 
-    if existing:
+    state = module.params['state']
-        input['id'] = existing['id']
+    project_identifier = module.params['project']
-
+    hook_url = module.params['hook_url']
-    changed = False
+    push_events = module.params['push_events']
-    success = True
+    issues_events = module.params['issues_events']
-    response = None
+    merge_requests_events = module.params['merge_requests_events']
-
+    tag_push_events = module.params['tag_push_events']
-    if state == 'present':
+    note_events = module.params['note_events']
-        if not existing or input['token'] or not _are_equivalent(existing, input):
+    job_events = module.params['job_events']
-            if not module.check_mode:
+    pipeline_events = module.params['pipeline_events']
-                success, response = _publish(module, api_url, project, input, access_token, private_token)
+    wiki_page_events = module.params['wiki_page_events']
-            changed = True
+    enable_ssl_verification = module.params['enable_ssl_verification']
+    hook_token = module.params['token']
+
+    if not HAS_GITLAB_PACKAGE:
+        module.fail_json(msg=missing_required_lib("python-gitlab"), exception=GITLAB_IMP_ERR)
+
+    try:
+        gitlab_instance = gitlab.Gitlab(url=gitlab_url, ssl_verify=validate_certs, email=gitlab_user, password=gitlab_password,
+                                        private_token=gitlab_token, api_version=4)
+        gitlab_instance.auth()
+    except (gitlab.exceptions.GitlabAuthenticationError, gitlab.exceptions.GitlabGetError) as e:
+        module.fail_json(msg="Failed to connect to Gitlab server: %s" % to_native(e))
+    except (gitlab.exceptions.GitlabHttpError) as e:
+        module.fail_json(msg="Failed to connect to Gitlab server: %s. \
+            Gitlab remove Session API now that private tokens are removed from user API endpoints since version 10.2." % to_native(e))
+
+    gitlab_hook = GitLabHook(module, gitlab_instance)
+
+    project = findProject(gitlab_instance, project_identifier)
+
+    if project is None:
+        module.fail_json(msg="Failed to create hook: project %s doesn't exists" % project_identifier)
+
+    hook_exists = gitlab_hook.existsHooks(project, hook_url)
+
+    if state == 'absent':
+        if hook_exists:
+            gitlab_hook.deleteHook()
+            module.exit_json(changed=True, msg="Successfully deleted hook %s" % hook_url)
         else:
-        if existing:
+            module.exit_json(changed=False, msg="Hook deleted or does not exists")
-            if not module.check_mode:
-                success, response = _delete(module, api_url, project, existing['id'], access_token, private_token)
-            changed = True
 
-    if success:
+    if state == 'present':
-        module.exit_json(changed=changed, msg='Success', result=response, previous_version=existing)
+        if gitlab_hook.createOrUpdateHook(project, hook_url, {
+                                          "push_events": push_events,
+                                          "issues_events": issues_events,
+                                          "merge_requests_events": merge_requests_events,
+                                          "tag_push_events": tag_push_events,
+                                          "note_events": note_events,
+                                          "job_events": job_events,
+                                          "pipeline_events": pipeline_events,
+                                          "wiki_page_events": wiki_page_events,
+                                          "enable_ssl_verification": enable_ssl_verification,
+                                          "token": hook_token}):
+
+            module.exit_json(changed=True, msg="Successfully created or updated the hook %s" % hook_url, hook=gitlab_hook.hookObject._attrs)
         else:
-        module.fail_json(msg='Failure', error=response)
+            module.exit_json(changed=False, msg="No need to update the hook %s" % hook_url, hook=gitlab_hook.hookObject._attrs)
 
 
 if __name__ == '__main__':

lib/ansible/modules/source_control/gitlab_project.py

@@ -1,16 +1,17 @@
 #!/usr/bin/python
-# (c) 2015, Werner Dijkerman (ikben@werner-dijkerman.nl)
+# -*- coding: utf-8 -*-
+
+# Copyright: (c) 2019, Guillaume Martinez (guillaume.lunik@gmail.com)
+# Copyright: (c) 2015, Werner Dijkerman (ikben@werner-dijkerman.nl)
 # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
 
 from __future__ import absolute_import, division, print_function
 __metaclass__ = type
 
-
 ANSIBLE_METADATA = {'metadata_version': '1.1',
                     'status': ['preview'],
                     'supported_by': 'community'}
 
-
 DOCUMENTATION = '''
 ---
 module: gitlab_project
@@ -20,114 +21,117 @@ description:
   - When the project does exists and state=absent, the project will be deleted.
   - When changes are made to the project, the project will be updated.
 version_added: "2.1"
-author: "Werner Dijkerman (@dj-wasabi)"
+author:
+  - Werner Dijkerman (@dj-wasabi)
+  - Guillaume Martinez (@Lunik)
 requirements:
-    - pyapi-gitlab python module
+  - python >= 2.7
+  - python-gitlab python module
+extends_documentation_fragment:
+    - auth_basic
 options:
   server_url:
     description:
-            - Url of Gitlab server, with protocol (http or https).
+      - The URL of the Gitlab server, with protocol (i.e. http or https).
     required: true
-    validate_certs:
+    type: str
-        description:
-            - When using https if SSL certificate needs to be verified.
-        type: bool
-        default: 'yes'
-        aliases:
-            - verify_ssl
   login_user:
     description:
       - Gitlab user name.
+    type: str
   login_password:
     description:
       - Gitlab password for login_user
-    login_token:
+    type: str
+  api_token:
     description:
       - Gitlab token for logging in.
+    type: str
+    aliases:
+      - login_token
   group:
     description:
-            - The name of the group of which this projects belongs to.
+      - Id or The full path of the group of which this projects belongs to.
-            - When not provided, project will belong to user which is configured in 'login_user' or 'login_token'
+    type: str
-            - When provided with username, project will be created for this user. 'login_user' or 'login_token' needs admin rights.
   name:
     description:
       - The name of the project
     required: true
+    type: str
   path:
     description:
       - The path of the project you want to create, this will be server_url/<group>/path
       - If not supplied, name will be used.
+    type: str
   description:
     description:
       - An description for the project.
+    type: str
   issues_enabled:
     description:
       - Whether you want to create issues or not.
       - Possible values are true and false.
     type: bool
-        default: 'yes'
+    default: yes
   merge_requests_enabled:
     description:
       - If merge requests can be made or not.
       - Possible values are true and false.
     type: bool
-        default: 'yes'
+    default: yes
   wiki_enabled:
     description:
       - If an wiki for this project should be available or not.
       - Possible values are true and false.
     type: bool
-        default: 'yes'
+    default: yes
   snippets_enabled:
     description:
       - If creating snippets should be available or not.
       - Possible values are true and false.
     type: bool
-        default: 'yes'
+    default: yes
-    public:
+  visibility:
-        description:
-            - If the project is public available or not.
-            - Setting this to true is same as setting visibility_level to 20.
-            - Possible values are true and false.
-        type: bool
-        default: 'no'
-    visibility_level:
     description:
-            - Private. visibility_level is 0. Project access must be granted explicitly for each user.
+      - Private. Project access must be granted explicitly for each user.
-            - Internal. visibility_level is 10. The project can be cloned by any logged in user.
+      - Internal. The project can be cloned by any logged in user.
-            - Public. visibility_level is 20. The project can be cloned without any authentication.
+      - Public. The project can be cloned without any authentication.
-            - Possible values are 0, 10 and 20.
+    default: private
-        default: 0
+    type: str
+    choices: ["private", "internal", "public"]
+    aliases:
+      - visibility_level
   import_url:
     description:
       - Git repository which will be imported into gitlab.
       - Gitlab server needs read access to this git repository.
-        type: bool
+    required: false
-        default: 'no'
+    type: str
   state:
     description:
       - create or delete project.
       - Possible values are present and absent.
-        default: "present"
+    default: present
+    type: str
     choices: ["present", "absent"]
 '''
 
 EXAMPLES = '''
 - name: Delete Gitlab Project
   gitlab_project:
-    server_url: http://gitlab.example.com
+    api_url: https://gitlab.example.com/
+    api_token: "{{ access_token }}"
     validate_certs: False
-    login_token: WnUzDsxjy8230-Dy_k
     name: my_first_project
     state: absent
   delegate_to: localhost
 
 - name: Create Gitlab Project in group Ansible
   gitlab_project:
-    server_url: https://gitlab.example.com
+    api_url: https://gitlab.example.com/
     validate_certs: True
-    login_user: dj-wasabi
+    api_username: dj-wasabi
-    login_password: MySecretPassword
+    api_password: "MySecretPassword"
     name: my_first_project
     group: ansible
     issues_enabled: False
@@ -138,257 +142,277 @@ EXAMPLES = '''
   delegate_to: localhost
 '''
 
-RETURN = '''# '''
+RETURN = '''
+msg:
+  description: Success or failure message
+  returned: always
+  type: str
+  sample: "Success"
+
+result:
+  description: json parsed response from the server
+  returned: always
+  type: dict
+
+error:
+  description: the error message returned by the Gitlab API
+  returned: failed
+  type: str
+  sample: "400: path is already in use"
+
+project:
+  description: API object
+  returned: always
+  type: dict
+'''
 
+import os
+import traceback
+
+GITLAB_IMP_ERR = None
 try:
     import gitlab
     HAS_GITLAB_PACKAGE = True
 except Exception:
+    GITLAB_IMP_ERR = traceback.format_exc()
     HAS_GITLAB_PACKAGE = False
 
-from ansible.module_utils.basic import AnsibleModule
+from ansible.module_utils.api import basic_auth_argument_spec
+from ansible.module_utils.basic import AnsibleModule, missing_required_lib
 from ansible.module_utils._text import to_native
 
+from ansible.module_utils.gitlab import findGroup, findProject
+
 
 class GitLabProject(object):
-    def __init__(self, module, git):
+    def __init__(self, module, gitlab_instance):
         self._module = module
-        self._gitlab = git
+        self._gitlab = gitlab_instance
+        self.projectObject = None
 
-    def createOrUpdateProject(self, project_exists, group_name, import_url, arguments):
+    '''
-        is_user = False
+    @param project_name Name of the project
-        group_id = self.getGroupId(group_name)
+    @param namespace Namespace Object (User or Group)
-        if not group_id:
+    @param options Options of the project
-            group_id = self.getUserId(group_name)
+    '''
-            is_user = True
+    def createOrUpdateProject(self, project_name, namespace, options):
-
+        changed = False
-        if project_exists:
+
-            # Edit project
+        # Because we have already call userExists in main()
-            return self.updateProject(group_name, arguments)
+        if self.projectObject is None:
+            project = self.createProject(namespace, {
+                'name': project_name,
+                'path': options['path'],
+                'description': options['description'],
+                'issues_enabled': options['issues_enabled'],
+                'merge_requests_enabled': options['merge_requests_enabled'],
+                'wiki_enabled': options['wiki_enabled'],
+                'snippets_enabled': options['snippets_enabled'],
+                'visibility': options['visibility'],
+                'import_url': options['import_url']})
+            changed = True
         else:
-            # Create project
+            changed, project = self.updateProject(self.projectObject, {
+                'name': project_name,
+                'description': options['description'],
+                'issues_enabled': options['issues_enabled'],
+                'merge_requests_enabled': options['merge_requests_enabled'],
+                'wiki_enabled': options['wiki_enabled'],
+                'snippets_enabled': options['snippets_enabled'],
+                'visibility': options['visibility']})
+
+        self.projectObject = project
+        if changed:
             if self._module.check_mode:
-                self._module.exit_json(changed=True)
+                self._module.exit_json(changed=True, msg="Successfully created or updated the project %s" % project_name)
-            return self.createProject(is_user, group_id, import_url, arguments)
 
-    def createProject(self, is_user, user_id, import_url, arguments):
+            try:
-        if is_user:
+                project.save()
-            result = self._gitlab.createprojectuser(user_id=user_id, import_url=import_url, **arguments)
+            except Exception as e:
-        else:
+                self._module.fail_json(msg="Failed update project: %s " % e)
-            group_id = user_id
-            result = self._gitlab.createproject(namespace_id=group_id, import_url=import_url, **arguments)
-
-        if not result:
-            self._module.fail_json(msg="Failed to create project %r" % arguments['name'])
-
-        return result
-
-    def deleteProject(self, group_name, project_name):
-        if self.existsGroup(group_name):
-            project_owner = group_name
-        else:
-            project_owner = self._gitlab.currentuser()['username']
-
-        search_results = self._gitlab.searchproject(search=project_name)
-        for result in search_results:
-            owner = result['namespace']['name']
-            if owner == project_owner:
-                return self._gitlab.deleteproject(result['id'])
-
-    def existsProject(self, group_name, project_name):
-        if self.existsGroup(group_name):
-            project_owner = group_name
-        else:
-            project_owner = self._gitlab.currentuser()['username']
-
-        search_results = self._gitlab.searchproject(search=project_name)
-        for result in search_results:
-            owner = result['namespace']['name']
-            if owner == project_owner:
             return True
+        else:
             return False
 
-    def existsGroup(self, group_name):
+    '''
-        if group_name is not None:
+    @param namespace Namespace Object (User or Group)
-            # Find the group, if group not exists we try for user
+    @param arguments Attributs of the project
-            for group in self._gitlab.getall(self._gitlab.getgroups):
+    '''
-                if group['name'] == group_name:
+    def createProject(self, namespace, arguments):
+        if self._module.check_mode:
             return True
 
-            user_name = group_name
+        arguments['namespace_id'] = namespace.id
-            user_data = self._gitlab.getusers(search=user_name)
+        try:
-            for data in user_data:
+            project = self._gitlab.projects.create(arguments)
-                if 'id' in user_data:
+        except (gitlab.exceptions.GitlabCreateError) as e:
-                    return True
+            self._module.fail_json(msg="Failed to create project: %s " % to_native(e))
-        return False
 
-    def getGroupId(self, group_name):
+        return project
-        if group_name is not None:
-            # Find the group, if group not exists we try for user
-            for group in self._gitlab.getall(self._gitlab.getgroups):
-                if group['name'] == group_name:
-                    return group['id']
 
-    def getProjectId(self, group_name, project_name):
+    '''
-        if self.existsGroup(group_name):
+    @param project Project Object
-            project_owner = group_name
+    @param arguments Attributs of the project
-        else:
+    '''
-            project_owner = self._gitlab.currentuser()['username']
+    def updateProject(self, project, arguments):
+        changed = False
 
-        search_results = self._gitlab.searchproject(search=project_name)
+        for arg_key, arg_value in arguments.items():
-        for result in search_results:
+            if arguments[arg_key] is not None:
-            owner = result['namespace']['name']
+                if getattr(project, arg_key) != arguments[arg_key]:
-            if owner == project_owner:
+                    setattr(project, arg_key, arguments[arg_key])
-                return result['id']
+                    changed = True
 
-    def getUserId(self, user_name):
+        return (changed, project)
-        user_data = self._gitlab.getusers(search=user_name)
 
-        for data in user_data:
+    def deleteProject(self):
-            if 'id' in data:
+        if self._module.check_mode:
-                return data['id']
+            return True
-        return self._gitlab.currentuser()['id']
 
-    def to_bool(self, value):
+        project = self.projectObject
-        if value:
-            return 1
-        else:
-            return 0
 
-    def updateProject(self, group_name, arguments):
+        return project.delete()
-        project_changed = False
-        project_name = arguments['name']
-        project_id = self.getProjectId(group_name, project_name)
-        project_data = self._gitlab.getproject(project_id=project_id)
 
-        for arg_key, arg_value in arguments.items():
+    '''
-            project_data_value = project_data[arg_key]
+    @param namespace User/Group object
+    @param name Name of the project
+    '''
+    def existsProject(self, namespace, path):
+        # When project exists, object will be stored in self.projectObject.
+        project = findProject(self._gitlab, namespace.full_path + '/' + path)
+        if project:
+            self.projectObject = project
+            return True
+        return False
 
-            if isinstance(project_data_value, bool) or project_data_value is None:
-                to_bool = self.to_bool(project_data_value)
-                if to_bool != arg_value:
-                    project_changed = True
-                    continue
-            else:
-                if project_data_value != arg_value:
-                    project_changed = True
 
-        if project_changed:
+def deprecation_warning(module):
-            if self._module.check_mode:
+    deprecated_aliases = ['login_token']
-                self._module.exit_json(changed=True)
+
-            return self._gitlab.editproject(project_id=project_id, **arguments)
+    module.deprecate("Aliases \'{aliases}\' are deprecated".format(aliases='\', \''.join(deprecated_aliases)), 2.10)
-        else:
-            return False
 
 
 def main():
-    module = AnsibleModule(
+    argument_spec = basic_auth_argument_spec()
-        argument_spec=dict(
+    argument_spec.update(dict(
-            server_url=dict(required=True),
+        server_url=dict(type='str', required=True, removed_in_version=2.10),
-            validate_certs=dict(required=False, default=True, type='bool', aliases=['verify_ssl']),
+        login_user=dict(type='str', no_log=True, removed_in_version=2.10),
-            login_user=dict(required=False, no_log=True),
+        login_password=dict(type='str', no_log=True, removed_in_version=2.10),
-            login_password=dict(required=False, no_log=True),
+        api_token=dict(type='str', no_log=True, aliases=["login_token"]),
-            login_token=dict(required=False, no_log=True),
+        group=dict(type='str'),
-            group=dict(required=False),
+        name=dict(type='str', required=True),
-            name=dict(required=True),
+        path=dict(type='str'),
-            path=dict(required=False),
+        description=dict(type='str'),
-            description=dict(required=False),
+        issues_enabled=dict(type='bool', default=True),
-            issues_enabled=dict(default=True, type='bool'),
+        merge_requests_enabled=dict(type='bool', default=True),
-            merge_requests_enabled=dict(default=True, type='bool'),
+        wiki_enabled=dict(type='bool', default=True),
-            wiki_enabled=dict(default=True, type='bool'),
         snippets_enabled=dict(default=True, type='bool'),
-            public=dict(default=False, type='bool'),
+        visibility=dict(type='str', default="private", choices=["internal", "private", "public"], aliases=["visibility_level"]),
-            visibility_level=dict(default="0", choices=["0", "10", "20"]),
+        import_url=dict(type='str'),
-            import_url=dict(required=False),
+        state=dict(type='str', default="present", choices=["absent", "present"]),
-            state=dict(default="present", choices=["present", 'absent']),
+    ))
-        ),
+
-        supports_check_mode=True
+    module = AnsibleModule(
+        argument_spec=argument_spec,
+        mutually_exclusive=[
+            ['api_url', 'server_url'],
+            ['api_username', 'login_user'],
+            ['api_password', 'login_password'],
+            ['api_username', 'api_token'],
+            ['api_password', 'api_token'],
+            ['login_user', 'login_token'],
+            ['login_password', 'login_token']
+        ],
+        required_together=[
+            ['api_username', 'api_password'],
+            ['login_user', 'login_password'],
+        ],
+        required_one_of=[
+            ['api_username', 'api_token', 'login_user', 'login_token']
+        ],
+        supports_check_mode=True,
     )
 
-    if not HAS_GITLAB_PACKAGE:
+    deprecation_warning(module)
-        module.fail_json(msg="Missing required gitlab module (check docs or install with: pip install pyapi-gitlab")
 
     server_url = module.params['server_url']
-    verify_ssl = module.params['validate_certs']
     login_user = module.params['login_user']
     login_password = module.params['login_password']
-    login_token = module.params['login_token']
+
-    group_name = module.params['group']
+    api_url = module.params['api_url']
+    validate_certs = module.params['validate_certs']
+    api_user = module.params['api_username']
+    api_password = module.params['api_password']
+
+    gitlab_url = server_url if api_url is None else api_url
+    gitlab_user = login_user if api_user is None else api_user
+    gitlab_password = login_password if api_password is None else api_password
+    gitlab_token = module.params['api_token']
+
+    group_identifier = module.params['group']
     project_name = module.params['name']
     project_path = module.params['path']
-    description = module.params['description']
+    project_description = module.params['description']
     issues_enabled = module.params['issues_enabled']
     merge_requests_enabled = module.params['merge_requests_enabled']
     wiki_enabled = module.params['wiki_enabled']
     snippets_enabled = module.params['snippets_enabled']
-    public = module.params['public']
+    visibility = module.params['visibility']
-    visibility_level = module.params['visibility_level']
     import_url = module.params['import_url']
     state = module.params['state']
 
-    # We need both login_user and login_password or login_token, otherwise we fail.
+    if not HAS_GITLAB_PACKAGE:
-    if login_user is not None and login_password is not None:
+        module.fail_json(msg=missing_required_lib("python-gitlab"), exception=GITLAB_IMP_ERR)
-        use_credentials = True
+
-    elif login_token is not None:
+    try:
-        use_credentials = False
+        gitlab_instance = gitlab.Gitlab(url=gitlab_url, ssl_verify=validate_certs, email=gitlab_user, password=gitlab_password,
-    else:
+                                        private_token=gitlab_token, api_version=4)
-        module.fail_json(msg="No login credentials are given. Use login_user with login_password, or login_token")
+        gitlab_instance.auth()
+    except (gitlab.exceptions.GitlabAuthenticationError, gitlab.exceptions.GitlabGetError) as e:
+        module.fail_json(msg="Failed to connect to Gitlab server: %s" % to_native(e))
+    except (gitlab.exceptions.GitlabHttpError) as e:
+        module.fail_json(msg="Failed to connect to Gitlab server: %s. \
+            Gitlab remove Session API now that private tokens are removed from user API endpoints since version 10.2." % to_native(e))
 
     # Set project_path to project_name if it is empty.
     if project_path is None:
         project_path = project_name.replace(" ", "_")
 
-    # Gitlab API makes no difference between upper and lower cases, so we lower them.
+    gitlab_project = GitLabProject(module, gitlab_instance)
-    project_name = project_name.lower()
-    project_path = project_path.lower()
-    if group_name is not None:
-        group_name = group_name.lower()
 
-    # Lets make an connection to the Gitlab server_url, with either login_user and login_password
+    if group_identifier:
-    # or with login_token
+        group = findGroup(gitlab_instance, group_identifier)
-    try:
+        if group is None:
-        if use_credentials:
+            module.fail_json(msg="Failed to create project: group %s doesn't exists" % group_identifier)
-            git = gitlab.Gitlab(host=server_url, verify_ssl=verify_ssl)
-            git.login(user=login_user, password=login_password)
-        else:
-            git = gitlab.Gitlab(server_url, token=login_token, verify_ssl=verify_ssl)
-    except Exception as e:
-        module.fail_json(msg="Failed to connect to Gitlab server: %s " % to_native(e))
 
-    # Check if user is authorized or not before proceeding to any operations
+        namespace = gitlab_instance.namespaces.get(group.id)
-    # if not, exit from here
+        project_exists = gitlab_project.existsProject(namespace, project_path)
-    auth_msg = git.currentuser().get('message', None)
+    else:
-    if auth_msg is not None and auth_msg == '401 Unauthorized':
+        user = gitlab_instance.users.list(username=gitlab_instance.user.username)[0]
-        module.fail_json(msg='User unauthorized',
+        namespace = gitlab_instance.namespaces.get(user.id)
-                         details="User is not allowed to access Gitlab server "
+        project_exists = gitlab_project.existsProject(namespace, project_path)
-                                 "using login_token. Please check login_token")
 
-    # Validate if project exists and take action based on "state"
+    if state == 'absent':
-    project = GitLabProject(module, git)
+        if project_exists:
-    project_exists = project.existsProject(group_name, project_name)
+            gitlab_project.deleteProject()
+            module.exit_json(changed=True, msg="Successfully deleted project %s" % project_name)
+        else:
+            module.exit_json(changed=False, msg="Project deleted or does not exists")
 
-    # Creating the project dict
+    if state == 'present':
-    arguments = {"name": project_name,
+        if gitlab_project.createOrUpdateProject(project_name, namespace, {
                                                 "path": project_path,
-                 "description": description,
+                                                "description": project_description,
-                 "issues_enabled": project.to_bool(issues_enabled),
+                                                "issues_enabled": issues_enabled,
-                 "merge_requests_enabled": project.to_bool(merge_requests_enabled),
+                                                "merge_requests_enabled": merge_requests_enabled,
-                 "wiki_enabled": project.to_bool(wiki_enabled),
+                                                "wiki_enabled": wiki_enabled,
-                 "snippets_enabled": project.to_bool(snippets_enabled),
+                                                "snippets_enabled": snippets_enabled,
-                 "public": project.to_bool(public),
+                                                "visibility": visibility,
-                 "visibility_level": int(visibility_level)}
+                                                "import_url": import_url}):
-
+
-    if project_exists and state == "absent":
+            module.exit_json(changed=True, msg="Successfully created or updated the project %s" % project_name, project=gitlab_project.projectObject._attrs)
-        project.deleteProject(group_name, project_name)
-        module.exit_json(changed=True, result="Successfully deleted project %s" % project_name)
-    else:
-        if state == "absent":
-            module.exit_json(changed=False, result="Project deleted or does not exist")
-        else:
-            if project.createOrUpdateProject(project_exists, group_name, import_url, arguments):
-                module.exit_json(changed=True, result="Successfully created or updated the project %s" % project_name)
         else:
-                module.exit_json(changed=False)
+            module.exit_json(changed=False, msg="No need to update the project %s" % project_name, project=gitlab_project.projectObject._attrs)
 
 
 if __name__ == '__main__':

lib/ansible/modules/source_control/gitlab_runner.py

@@ -1,16 +1,16 @@
 #!/usr/bin/python
+# -*- coding: utf-8 -*-
 
+# Copyright: (c) 2019, Guillaume Martinez (guillaume.lunik@gmail.com)
 # Copyright: (c) 2018, Samy Coenen <samy.coenen@nubera.be>
 # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
 
 from __future__ import absolute_import, division, print_function
 __metaclass__ = type
 
-ANSIBLE_METADATA = {
+ANSIBLE_METADATA = {'metadata_version': '1.1',
-    'metadata_version': '1.1',
                     'status': ['preview'],
-    'supported_by': 'community'
+                    'supported_by': 'community'}
-}
 
 DOCUMENTATION = '''
 ---
@@ -30,29 +30,39 @@ notes:
   - To create a new runner at least the C(private_token), C(registration_token), C(name) and C(url) options are required.
   - Runners need to have unique names.
 version_added: 2.8
-author: "Samy Coenen (@SamyCoenen)"
+author:
+  - Samy Coenen (@SamyCoenen)
+  - Guillaume Martinez (@Lunik)
+requirements:
+  - python >= 2.7
+  - python-gitlab python module
+extends_documentation_fragment:
+    - auth_basic
 options:
-    private_token:
+  url:
+    description:
+      - The URL of the Gitlab server, with protocol (i.e. http or https).
+    required: true
+    type: str
+  api_token:
     description:
       - Your private token to interact with the GitLab API.
     required: True
     type: str
-    name:
+    aliases:
+      - private_token
+  description:
     description:
       - The unique name of the runner.
     required: True
     type: str
-    api_timeout:
+    aliases:
-        description:
+      - name
-            - The maximum time that a request will be attempted to the GitLab API.
-        required: False
-        default: 30
-        type: int
   state:
     description:
       - Make sure that the runner with the same name exists with the same configuration or delete the runner with the same name.
     required: False
-        default: "present"
+    default: present
     choices: ["present", "absent"]
     type: str
   registration_token:
@@ -60,31 +70,23 @@ options:
       - The registration token is used to register new runners.
     required: True
     type: str
-    url:
-        description:
-            - The GitLab URL including the API v4 path and http or https.
-        required: False
-        default: "https://gitlab.com/api/v4/"
-        type: str
   active:
     description:
       - Define if the runners is immediately active after creation.
     required: False
-        default: True
+    default: yes
-        choices: [True, False]
     type: bool
   locked:
     description:
       - Determines if the runner is locked or not.
     required: False
     default: False
-        choices: [true, false]
     type: bool
   access_level:
     description:
       - Determines if a runner can pick up jobs from protected branches.
     required: False
-        default: "ref_protected"
+    default: ref_protected
     choices: ["ref_protected", "not_protected"]
     type: str
   maximum_timeout:
@@ -97,347 +99,291 @@ options:
     description:
       - Run untagged jobs or not.
     required: False
-        default: True
+    default: yes
     type: bool
   tag_list:
     description: The tags that apply to the runner.
     required: False
-        default: ["docker"]
+    default: []
     type: list
 '''
 
 EXAMPLES = '''
-# Register a new runner (if it does not exist)
+- name: "Register runner"
-
-- name: Register runner
   gitlab_runner:
-    url: https://gitlab.com/api/v4/
+    api_url: https://gitlab.example.com/
-    private_token: ...5432632464326432632463246...
+    api_token: "{{ access_token }}"
-    registration_token: ...4gfdsg345...
+    registration_token: 4gfdsg345
-    name: Docker Machine t1
+    description: Docker Machine t1
     state: present
     active: True
     tag_list: ['docker']
     run_untagged: False
     locked: False
-    access_level: ref_protected
-  register: api
 
-# Delete a runner
+- name: "Delete runner"
-- name: Delete runner
   gitlab_runner:
-    url: https://gitlab.com/api/v4/
+    api_url: https://gitlab.example.com/
-    private_token: ...5432632464326432632463246...
+    api_token: "{{ access_token }}"
-    registration_token: ...4gfdsg345...
+    description: Docker Machine t1
-    name: Docker Machine t1
     state: absent
-  register: api
 '''
 
 RETURN = '''
-changed:
-    description: Values changed on the API
-    returned: changed
-    type: bool
-    sample: false
 msg:
-    description: Information returned from the API when updating a runner, a create only returns the id and token.
+  description: Success or failure message
+  returned: always
+  type: str
+  sample: "Success"
+
+result:
+  description: json parsed response from the server
   returned: always
   type: dict
-    sample: {
+
-      "id": 31,
+error:
-      "description": "Docker Machine t2",
+  description: the error message returned by the Gitlab API
-      "active": true,
+  returned: failed
-      "is_shared": true,
-      "name": null,
-      "online": null,
-      "status": "not_connected",
-      "tag_list": [
-          "docker"
-      ],
-      "run_untagged": true,
-      "locked": false,
-      "maximum_timeout": null,
-      "access_level": "ref_protected",
-      "version": null,
-      "revision": null,
-      "platform":null,
-      "architecture": null,
-      "contacted_at": null,
-      "token": "ba4be.....e6a3b8",
-      "projects": [],
-      "groups": []
-    }
-token:
-    description: Runner token of affected runner
-    returned: when registered or updated runner
   type: str
-    sample: ["2a5aeecc61dc98c4d780b14b330e3282"]
+  sample: "400: path is already in use"
+
+runner:
+  description: API object
+  returned: always
+  type: dict
 '''
 
-from ansible.module_utils.basic import env_fallback
+import os
-from ansible.module_utils.basic import AnsibleModule
+import re
-from ansible.module_utils.urls import fetch_url
+import traceback
-import json
+
+GITLAB_IMP_ERR = None
+try:
+    import gitlab
+    HAS_GITLAB_PACKAGE = True
+except Exception:
+    GITLAB_IMP_ERR = traceback.format_exc()
+    HAS_GITLAB_PACKAGE = False
+
+from ansible.module_utils.api import basic_auth_argument_spec
+from ansible.module_utils.basic import AnsibleModule, missing_required_lib
+from ansible.module_utils._text import to_native
+
+try:
+    cmp
+except NameError:
+    def cmp(a, b):
+        return (a > b) - (a < b)
 
 
-class AnsibleGitlabAPI(AnsibleModule):
+class GitLabRunner(object):
-    def __init__(self, module, url, private_token):
+    def __init__(self, module, gitlab_instance):
         self._module = module
-        self._auth_header = {'PRIVATE-TOKEN': private_token}
+        self._gitlab = gitlab_instance
-        self.url = url
+        self.runnerObject = None
-        self.timeout = module.params['api_timeout']
+
-
+    def createOrUpdateRunner(self, description, options):
-    def check_response(self, info, response, api_call):
+        changed = False
-        """
+
-        Checks response code.
+        # Because we have already call userExists in main()
-        Returns: response in JSON.
+        if self.runnerObject is None:
-        """
+            runner = self.createRunner({
-        if info['status'] in (200, 201):
+                'description': description,
-            return json.loads(response.read())
+                'active': options['active'],
-        elif info['status'] == 204:
+                'token': options['registration_token'],
-            return json.loads('{"msg":"Request completed"}')
+                'locked': options['locked'],
-        elif info['status'] in (403, 404):
+                'run_untagged': options['run_untagged'],
-            return None
+                'maximum_timeout': options['maximum_timeout'],
+                'tag_list': options['tag_list']})
+            changed = True
         else:
-            self._module.fail_json(msg='Failure while calling the GitLab API for '
+            changed, runner = self.updateRunner(self.runnerObject, {
-                                       '"%s".' % api_call, fetch_url_info=info)
+                'active': options['active'],
-
+                'locked': options['locked'],
-    def _get(self, api_call):
+                'run_untagged': options['run_untagged'],
-        resp, info = fetch_url(self._module, self.url + api_call,
+                'maximum_timeout': options['maximum_timeout'],
-                               headers=self._auth_header,
+                'access_level': options['access_level'],
-                               timeout=self.timeout)
+                'tag_list': options['tag_list']})
-        return self.check_response(info, resp, api_call)
+
-
+        self.runnerObject = runner
-    def _post(self, api_call, data=None):
+        if changed:
-        """
+            if self._module.check_mode:
-        Sends POST request.
+                self._module.exit_json(changed=True, msg="Successfully created or updated the runner %s" % description)
-        Returns: response.
+
-        """
+            try:
-        headers = self._auth_header.copy()
+                runner.save()
-        if data is not None:
+            except Exception as e:
-            data = self._module.jsonify(data)
+                self._module.fail_json(msg="Failed to update runner: %s " % to_native(e))
-            headers['Content-type'] = 'application/json'
+            return True
-
+        else:
-        resp, info = fetch_url(self._module,
+            return False
-                               self.url + api_call,
+
-                               headers=headers,
+    '''
-                               method='POST',
+    @param arguments Attributs of the runner
-                               data=data,
+    '''
-                               timeout=self.timeout)
+    def createRunner(self, arguments):
-        return self.check_response(info, resp, api_call)
+        if self._module.check_mode:
-
+            return True
-    def _put(self, api_call, data=None):
+
-        """
+        try:
-        Sends PUT request.
+            runner = self._gitlab.runners.create(arguments)
-        Returns: response.
+        except (gitlab.exceptions.GitlabCreateError) as e:
-        """
+            self._module.fail_json(msg="Failed to create runner: %s " % to_native(e))
-        headers = self._auth_header.copy()
+
-        if data is not None:
+        return runner
-            data = self._module.jsonify(data)
+
-            headers['Content-type'] = 'application/json'
+    '''
-
+    @param runner Runner object
-        resp, info = fetch_url(self._module,
+    @param arguments Attributs of the runner
-                               self.url + api_call,
+    '''
-                               headers=headers,
+    def updateRunner(self, runner, arguments):
-                               method='PUT',
+        changed = False
-                               data=data,
+
-                               timeout=self.timeout)
+        for arg_key, arg_value in arguments.items():
-        return self.check_response(info, resp, api_call)
+            if arguments[arg_key] is not None:
-
+                if isinstance(arguments[arg_key], list):
-    def _delete(self, api_call):
+                    list1 = getattr(runner, arg_key)
-        """
+                    list1.sort()
-        Sends DELETE request.
+                    list2 = arguments[arg_key]
-        Returns: response.
+                    list2.sort()
-        """
+                    if cmp(list1, list2):
-        resp, info = fetch_url(self._module,
+                        setattr(runner, arg_key, arguments[arg_key])
-                               self.url + api_call,
+                        changed = True
-                               headers=self._auth_header,
+                else:
-                               method='DELETE',
+                    if getattr(runner, arg_key) != arguments[arg_key]:
-                               timeout=self.timeout)
+                        setattr(runner, arg_key, arguments[arg_key])
-        return self.check_response(info, resp, api_call)
+                        changed = True
-
+
-    def get_runner_id(self, description):
+        return (changed, runner)
-        """
+
-        Gets the ID for a given description.
+    '''
-        Returns: ID as int.
+    @param description Description of the runner
-        """
+    '''
-        r = self._get('runners/all')
+    def findRunner(self, description):
-        if r is None:
+        runners = self._gitlab.runners.all()
-            return None
+        for runner in runners:
-        for runner in r:
+            if (runner['description'] == description):
-            if runner['description'] == description:
+                return self._gitlab.runners.get(runner['id'])
-                return runner.get('id')
+
-        return None
+    '''
-
+    @param description Description of the runner
-    def delete_runner(self, id):
+    '''
-        """
+    def existsRunner(self, description):
-        Sends DELETE request for runner with the same ID.
+        # When runner exists, object will be stored in self.runnerObject.
-        Returns: response.
+        runner = self.findRunner(description)
-        """
+
-        return self._delete('runners/' + str(id))
+        if runner:
-
+            self.runnerObject = runner
-    def update_runner(self, id, runner):
+            return True
-        """
+        return False
-        Sends UPDATE request for runner to change all the fields.
+
-        Returns: response.
+    def deleteRunner(self):
-        """
+        if self._module.check_mode:
-        form_data = runner.get_dict()
+            return True
-        return self._put('runners/' + str(id), form_data)
+
-
+        runner = self.runnerObject
-    def register_runner(self, runner, registration_token):
+
-        """
+        return runner.delete()
-        Sends POST request to register runner.
+
-        Returns: response.
+
-        """
+def deprecation_warning(module):
-        form_data = runner.get_dict()
+    deprecated_aliases = ['login_token']
-        form_data["token"] = registration_token
+
-        return self._post('runners/', form_data)
+    module.deprecate("Aliases \'{aliases}\' are deprecated".format(aliases='\', \''.join(deprecated_aliases)), 2.10)
-
-    def get_runner_list_short(self):
-        """
-        Sends GET request to get a global list of all runners.
-        Returns: JSON list.
-        """
-        return self._get('runners/all')
-
-    def get_runner_list(self, ids, primary_key):
-        """
-        Sends GET requests to get details of all runners.
-        Returns: JSON list.
-        """
-        d = {}
-        for i in ids:
-            runner_details = self.get_runner_details(i)
-            d[runner_details.get(primary_key)] = runner_details
-        return d
-
-    def get_runner_details(self, id):
-        """
-        Sends GET request to get the details of a specific runner .
-        Returns: JSON list.
-        """
-        return self._get('runners/' + str(id))
-
-
-class Runner():
-    def __init__(self, id, description, active, tag_list, run_untagged, locked, access_level, maximum_timeout):
-        self.id = id
-        self.description = description
-        self.active = active
-        self.tag_list = tag_list
-        self.run_untagged = run_untagged
-        self.locked = locked
-        self.access_level = access_level
-        self.maximum_timeout = maximum_timeout
-
-    def __eq__(self, other):
-        """
-        Compare every field and its value with the fields and values of another instance of this class.
-        Returns: boolean.
-        """
-        return self.__dict__ == other.__dict__
-
-    def get_dict(self):
-        """
-        Returns every field and its value of this class.
-        Returns: dict.
-        """
-        return self.__dict__
-
-
-def get_gitlab_argument_spec():
-    """
-    Returns argument spec with all optional or required Ansible arguments.
-    Returns: dict.
-    """
-    return dict(
-        private_token=dict(
-            fallback=(env_fallback, ['GITLAB_PRIVATE_TOKEN']),
-            no_log=True,
-            required=True),
-        name=dict(required=True, type='str'),
-        active=dict(required=False, type='bool', default=True, choices=[True, False]),
-        tag_list=dict(required=False, type='list',
-                      default=["docker"]),
-        run_untagged=dict(
-            required=False, type='bool', default=True),
-        locked=dict(required=False, type='bool', default=False, choices=[True, False]),
-        access_level=dict(required=False, type='str',
-                          default='ref_protected', choices=["ref_protected", "not_protected"]),
-        maximum_timeout=dict(
-            required=False, type='int', default=3600),
-        api_timeout=dict(default=30, type='int'),
-        url=dict(required=False, type='str',
-                 default="https://gitlab.com/api/v4/"),
-        registration_token=dict(required=True, type='str'),
-        state=dict(required=False, type='str', default="present", choices=["present", "absent"]),
-    )
 
 
 def main():
-    argument_spec = get_gitlab_argument_spec()
+    argument_spec = basic_auth_argument_spec()
+    argument_spec.update(dict(
+        url=dict(type='str', required=True, removed_in_version=2.10),
+        api_token=dict(type='str', no_log=True, aliases=["private_token"]),
+        description=dict(type='str', required=True, aliases=["name"]),
+        active=dict(type='bool', default=True),
+        tag_list=dict(type='list', default=[]),
+        run_untagged=dict(type='bool', default=True),
+        locked=dict(type='bool', default=False),
+        access_level=dict(type='str', default='ref_protected', choices=["not_protected", "ref_protected"]),
+        maximum_timeout=dict(type='int', default=3600),
+        registration_token=dict(type='str', required=True),
+        state=dict(type='str', default="present", choices=["absent", "present"]),
+    ))
+
     module = AnsibleModule(
         argument_spec=argument_spec,
+        mutually_exclusive=[
+            ['api_url', 'url'],
+            ['api_username', 'api_token'],
+            ['api_password', 'api_token'],
+        ],
+        required_together=[
+            ['api_username', 'api_password'],
+            ['login_user', 'login_password'],
+        ],
+        required_one_of=[
+            ['api_username', 'api_token']
+        ],
         supports_check_mode=True,
     )
 
-    target_state = module.params['state']
+    deprecation_warning(module)
-    description = module.params['name']
+
-    private_token = module.params['private_token']
+    url = re.sub('/api.*', '', module.params['url'])
-    url = module.params['url']
+
-    active = module.params['active']
+    api_url = module.params['api_url']
+    validate_certs = module.params['validate_certs']
+
+    gitlab_url = url if api_url is None else api_url
+    gitlab_user = module.params['api_username']
+    gitlab_password = module.params['api_password']
+    gitlab_token = module.params['api_token']
+
+    state = module.params['state']
+    runner_description = module.params['description']
+    runner_active = module.params['active']
     tag_list = module.params['tag_list']
     run_untagged = module.params['run_untagged']
-    locked = module.params['locked']
+    runner_locked = module.params['locked']
     access_level = module.params['access_level']
     maximum_timeout = module.params['maximum_timeout']
-
+    registration_token = module.params['registration_token']
-    api = AnsibleGitlabAPI(module, url, private_token)
+
-    id = api.get_runner_id(description)
+    if not HAS_GITLAB_PACKAGE:
-    target_runner = Runner(id, description, active, tag_list,
+        module.fail_json(msg=missing_required_lib("python-gitlab"), exception=GITLAB_IMP_ERR)
-                           run_untagged, locked, access_level, maximum_timeout)
+
-    token = None
+    try:
-    api_runner = None
+        gitlab_instance = gitlab.Gitlab(url=gitlab_url, ssl_verify=validate_certs, email=gitlab_user, password=gitlab_password,
-    response = None
+                                        private_token=gitlab_token, api_version=4)
-    changed = False
+        gitlab_instance.auth()
-
+    except (gitlab.exceptions.GitlabAuthenticationError, gitlab.exceptions.GitlabGetError) as e:
-    # Check if runner needs to be registered, updated or deleted
+        module.fail_json(msg="Failed to connect to Gitlab server: %s" % to_native(e))
-    # Don't actually change anything if module is in check_mode (dry run)
+    except (gitlab.exceptions.GitlabHttpError) as e:
-    if target_state == 'present':
+        module.fail_json(msg="Failed to connect to Gitlab server: %s. \
-        if id is None:
+            Gitlab remove Session API now that private tokens are removed from user API endpoints since version 10.2" % to_native(e))
-            if not module.check_mode:
+
-                response = api.register_runner(
+    gitlab_runner = GitLabRunner(module, gitlab_instance)
-                    target_runner, module.params["registration_token"])
+    runner_exists = gitlab_runner.existsRunner(runner_description)
-            token = response['token']
+
-            changed = True
+    if state == 'absent':
+        if runner_exists:
+            gitlab_runner.deleteRunner()
+            module.exit_json(changed=True, msg="Successfully deleted runner %s" % runner_description)
         else:
-            api_runner_details = api.get_runner_details(id)
+            module.exit_json(changed=False, msg="Runner deleted or does not exists")
-            response = api_runner_details
+
-            token = api_runner_details['token']
+    if state == 'present':
-            id = api_runner_details['id']
+        if gitlab_runner.createOrUpdateRunner(runner_description, {
-            description = api_runner_details['description']
+                                              "active": runner_active,
-            active = api_runner_details['active']
+                                              "tag_list": tag_list,
-            tag_list = api_runner_details['tag_list']
+                                              "run_untagged": run_untagged,
-            locked = api_runner_details['locked']
+                                              "locked": runner_locked,
-            access_level = api_runner_details['access_level']
+                                              "access_level": access_level,
-            maximum_timeout = api_runner_details['maximum_timeout']
+                                              "maximum_timeout": maximum_timeout,
-            api_runner = Runner(id, description, active, tag_list,
+                                              "registration_token": registration_token}):
-                                run_untagged, locked, access_level, maximum_timeout)
+            module.exit_json(changed=True, runner=gitlab_runner.runnerObject._attrs,
-            if api_runner == target_runner:
+                             msg="Successfully created or updated the runner %s" % runner_description)
-                changed = False
         else:
-                if not module.check_mode:
+            module.exit_json(changed=False, runner=gitlab_runner.runnerObject._attrs,
-                    response = api.update_runner(id, target_runner)
+                             msg="No need to update the runner %s" % runner_description)
-                changed = True
-    elif target_state == 'absent':
-        if id is None:
-            changed = False
-        else:
-            if not module.check_mode:
-                response = api.delete_runner(id)
-            changed = True
-    module.exit_json(changed=changed, msg=response, token=token)
 
 
 if __name__ == '__main__':

lib/ansible/modules/source_control/gitlab_user.py

@@ -1,16 +1,17 @@
 #!/usr/bin/python
-# (c) 2015, Werner Dijkerman (ikben@werner-dijkerman.nl)
+# -*- coding: utf-8 -*-
+
+# Copyright: (c) 2019, Guillaume Martinez (guillaume.lunik@gmail.com)
+# Copyright: (c) 2015, Werner Dijkerman (ikben@werner-dijkerman.nl)
 # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
 
 from __future__ import absolute_import, division, print_function
 __metaclass__ = type
 
-
 ANSIBLE_METADATA = {'metadata_version': '1.1',
                     'status': ['preview'],
                     'supported_by': 'community'}
 
-
 DOCUMENTATION = '''
 ---
 module: gitlab_user
@@ -20,95 +21,124 @@ description:
   - When the user does exists and state=absent, the user will be deleted.
   - When changes are made to user, the user will be updated.
 version_added: "2.1"
-author: "Werner Dijkerman (@dj-wasabi)"
+author:
+  - Werner Dijkerman (@dj-wasabi)
+  - Guillaume Martinez (@Lunik)
 requirements:
-    - pyapi-gitlab python module
+  - python >= 2.7
+  - python-gitlab python module
   - administrator rights on the Gitlab server
+extends_documentation_fragment:
+    - auth_basic
 options:
   server_url:
     description:
-            - Url of Gitlab server, with protocol (http or https).
+      - The URL of the Gitlab server, with protocol (i.e. http or https).
     required: true
-    validate_certs:
+    type: str
-        description:
-            - When using https if SSL certificate needs to be verified.
-        type: bool
-        default: 'yes'
-        aliases:
-            - verify_ssl
   login_user:
     description:
       - Gitlab user name.
+    type: str
   login_password:
     description:
       - Gitlab password for login_user
-    login_token:
+    type: str
+  api_token:
     description:
       - Gitlab token for logging in.
+    type: str
+    aliases:
+      - login_token
   name:
     description:
       - Name of the user you want to create
     required: true
+    type: str
   username:
     description:
       - The username of the user.
     required: true
+    type: str
   password:
     description:
       - The password of the user.
       - GitLab server enforces minimum password length to 8, set this value with 8 or more characters.
     required: true
+    type: str
   email:
     description:
       - The email that belongs to the user.
     required: true
+    type: str
   sshkey_name:
     description:
       - The name of the sshkey
+    type: str
   sshkey_file:
     description:
       - The ssh key itself.
+    type: str
   group:
     description:
+      - Id or Full path of parent group in the form of group/name
       - Add user as an member to this group.
+    type: str
   access_level:
     description:
       - The access level to the group. One of the following can be used.
       - guest
       - reporter
       - developer
-            - master
+      - master (alias for maintainer)
+      - maintainer
       - owner
+    default: guest
+    type: str
+    choices: ["guest", "reporter", "developer", "master", "maintainer", "owner"]
   state:
     description:
       - create or delete group.
       - Possible values are present and absent.
     default: present
+    type: str
     choices: ["present", "absent"]
   confirm:
     description:
       - Require confirmation.
     type: bool
-        default: 'yes'
+    default: yes
     version_added: "2.4"
+  isadmin:
+    description:
+      - Grant admin privilieges to the user
+    type: bool
+    default: no
+    version_added: "2.8"
+  external:
+    description:
+      - Define external parameter for this user
+    type: bool
+    default: no
+    version_added: "2.8"
 '''
 
 EXAMPLES = '''
-- name: Delete Gitlab User
+- name: "Delete Gitlab User"
   gitlab_user:
-    server_url: http://gitlab.example.com
+    api_url: https://gitlab.example.com/
+    api_token: "{{ access_token }}"
     validate_certs: False
-    login_token: WnUzDsxjy8230-Dy_k
     username: myusername
     state: absent
   delegate_to: localhost
 
-- name: Create Gitlab User
+- name: "Create Gitlab User"
   gitlab_user:
-    server_url: https://gitlab.dj-wasabi.local
+    api_url: https://gitlab.example.com/
     validate_certs: True
-    login_user: dj-wasabi
+    api_username: dj-wasabi
-    login_password: MySecretPassword
+    api_password: "MySecretPassword"
     name: My Name
     username: myusername
     password: mysecretpassword
@@ -116,233 +146,383 @@ EXAMPLES = '''
     sshkey_name: MySSH
     sshkey_file: ssh-rsa AAAAB3NzaC1yc...
     state: present
+    group: super_group/mon_group
+    access_level: owner
   delegate_to: localhost
 '''
 
-RETURN = '''# '''
+RETURN = '''
+msg:
+  description: Success or failure message
+  returned: always
+  type: str
+  sample: "Success"
+
+result:
+  description: json parsed response from the server
+  returned: always
+  type: dict
+
+error:
+  description: the error message returned by the Gitlab API
+  returned: failed
+  type: str
+  sample: "400: path is already in use"
+
+user:
+  description: API object
+  returned: always
+  type: dict
+'''
+
+import os
+import re
+import traceback
 
+GITLAB_IMP_ERR = None
 try:
     import gitlab
     HAS_GITLAB_PACKAGE = True
 except Exception:
+    GITLAB_IMP_ERR = traceback.format_exc()
     HAS_GITLAB_PACKAGE = False
 
-from ansible.module_utils.basic import AnsibleModule
+from ansible.module_utils.api import basic_auth_argument_spec
+from ansible.module_utils.basic import AnsibleModule, missing_required_lib
 from ansible.module_utils._text import to_native
 
+from ansible.module_utils.gitlab import findGroup
+
 
 class GitLabUser(object):
-    def __init__(self, module, git):
+    def __init__(self, module, gitlab_instance):
         self._module = module
-        self._gitlab = git
+        self._gitlab = gitlab_instance
-
+        self.userObject = None
-    def addToGroup(self, group_id, user_id, access_level):
+        self.ACCESS_LEVEL = {
-        if access_level == "guest":
+            'guest': gitlab.GUEST_ACCESS,
-            level = 10
+            'reporter': gitlab.REPORTER_ACCESS,
-        elif access_level == "reporter":
+            'developer': gitlab.DEVELOPER_ACCESS,
-            level = 20
+            'master': gitlab.MAINTAINER_ACCESS,
-        elif access_level == "developer":
+            'maintainer': gitlab.MAINTAINER_ACCESS,
-            level = 30
+            'owner': gitlab.OWNER_ACCESS}
-        elif access_level == "master":
+
-            level = 40
+    '''
-        elif access_level == "owner":
+    @param username Username of the user
-            level = 50
+    @param options User options
-        return self._gitlab.addgroupmember(group_id, user_id, level)
+    '''
-
+    def createOrUpdateUser(self, username, options):
-    def createOrUpdateUser(self, user_name, user_username, user_password, user_email, user_sshkey_name, user_sshkey_file, group_name, access_level, confirm):
+        changed = False
-        group_id = ''
+
-        arguments = {"name": user_name,
+        # Because we have already call userExists in main()
-                     "username": user_username,
+        if self.userObject is None:
-                     "email": user_email}
+            user = self.createUser({
-
+                'name': options['name'],
-        if group_name is not None:
+                'username': username,
-            if self.existsGroup(group_name):
+                'password': options['password'],
-                group_id = self.getGroupId(group_name)
+                'email': options['email'],
-
+                'skip_confirmation': not options['confirm'],
-        if self.existsUser(user_username):
+                'admin': options['isadmin'],
-            self.updateUser(group_id, user_sshkey_name, user_sshkey_file, access_level, arguments)
+                'external': options['external']})
+            changed = True
         else:
+            changed, user = self.updateUser(self.userObject, {
+                'name': options['name'],
+                'email': options['email'],
+                'is_admin': options['isadmin'],
+                'external': options['external']})
+
+        # Assign ssh keys
+        if options['sshkey_name'] and options['sshkey_file']:
+            changed = changed or self.addSshKeyToUser(user, {
+                'name': options['sshkey_name'],
+                'file': options['sshkey_file']})
+
+        # Assign group
+        if options['group_path']:
+            changed = changed or self.assignUserToGroup(user, options['group_path'], options['access_level'])
+
+        self.userObject = user
+        if changed:
             if self._module.check_mode:
-                self._module.exit_json(changed=True)
+                self._module.exit_json(changed=True, msg="Successfully created or updated the user %s" % username)
-            self.createUser(group_id, user_password, user_sshkey_name, user_sshkey_file, access_level, confirm, arguments)
+
-
+            try:
-    def createUser(self, group_id, user_password, user_sshkey_name, user_sshkey_file, access_level, confirm, arguments):
+                user.save()
-        user_changed = False
+            except Exception as e:
-
+                self._module.fail_json(msg="Failed to update user: %s " % to_native(e))
-        # Create the user
+            return True
-        user_username = arguments['username']
-        if self._gitlab.createuser(password=user_password, confirm=confirm, **arguments):
-            user_id = self.getUserId(user_username)
-            if self._gitlab.addsshkeyuser(user_id=user_id, title=user_sshkey_name, key=user_sshkey_file):
-                user_changed = True
-            # Add the user to the group if group_id is not empty
-            if group_id != '':
-                if self.addToGroup(group_id, user_id, access_level):
-                    user_changed = True
-            user_changed = True
-
-        # Exit with change to true or false
-        if user_changed:
-            self._module.exit_json(changed=True, result="Created the user")
         else:
-            self._module.exit_json(changed=False)
+            return False
 
-    def deleteUser(self, user_username):
+    '''
-        user_id = self.getUserId(user_username)
+    @param group User object
+    '''
+    def getUserId(self, user):
+        if user is not None:
+            return user.id
+        return None
 
-        if self._gitlab.deleteuser(user_id):
+    '''
-            self._module.exit_json(changed=True, result="Successfully deleted user %s" % user_username)
+    @param user User object
-        else:
+    @param sshkey_name Name of the ssh key
-            self._module.exit_json(changed=False, result="User %s already deleted or something went wrong" % user_username)
+    '''
+    def sshKeyExists(self, user, sshkey_name):
+        keyList = map(lambda k: k.title, user.keys.list())
+
+        return sshkey_name in keyList
 
-    def existsGroup(self, group_name):
+    '''
-        for group in self._gitlab.getall(self._gitlab.getgroups):
+    @param user User object
-            if group['name'] == group_name:
+    @param sshkey Dict containing sshkey infos {"name": "", "file": ""}
+    '''
+    def addSshKeyToUser(self, user, sshkey):
+        if not self.sshKeyExists(user, sshkey['name']):
+            if self._module.check_mode:
                 return True
-        return False
 
-    def existsUser(self, username):
+            try:
-        found_user = self._gitlab.getusers(search=username)
+                user.keys.create({
-        for user in found_user:
+                    'title': sshkey['name'],
-            if user['id'] != '':
+                    'key': sshkey['file']})
+            except gitlab.exceptions.GitlabCreateError as e:
+                self._module.fail_json(msg="Failed to assign sshkey to user: %s" % to_native(e))
             return True
         return False
 
-    def getGroupId(self, group_name):
+    '''
-        for group in self._gitlab.getall(self._gitlab.getgroups):
+    @param group Group object
-            if group['name'] == group_name:
+    @param user_id Id of the user to find
-                return group['id']
+    '''
+    def findMember(self, group, user_id):
+        try:
+            member = group.members.get(user_id)
+        except gitlab.exceptions.GitlabGetError as e:
+            return None
+        return member
+
+    '''
+    @param group Group object
+    @param user_id Id of the user to check
+    '''
+    def memberExists(self, group, user_id):
+        member = self.findMember(group, user_id)
+
+        return member is not None
 
-    def getUserId(self, username):
+    '''
-        found_user = self._gitlab.getusers(search=username)
+    @param group Group object
-        for user in found_user:
+    @param user_id Id of the user to check
-            if user['id'] != '':
+    @param access_level Gitlab access_level to check
-                return user['id']
+    '''
+    def memberAsGoodAccessLevel(self, group, user_id, access_level):
+        member = self.findMember(group, user_id)
 
-    def updateUser(self, group_id, user_sshkey_name, user_sshkey_file, access_level, arguments):
+        return member.access_level == access_level
-        user_changed = False
-        user_username = arguments['username']
-        user_id = self.getUserId(user_username)
-        user_data = self._gitlab.getuser(user_id=user_id)
 
-        # Lets check if we need to update the user
+    '''
-        for arg_key, arg_value in arguments.items():
+    @param user User object
-            if user_data[arg_key] != arg_value:
+    @param group_path Complete path of the Group including parent group path. <parent_path>/<group_path>
-                user_changed = True
+    @param access_level Gitlab access_level to assign
+    '''
+    def assignUserToGroup(self, user, group_identifier, access_level):
+        group = findGroup(self._gitlab, group_identifier)
 
-        if user_changed:
         if self._module.check_mode:
-                self._module.exit_json(changed=True)
+            return True
-            self._gitlab.edituser(user_id=user_id, **arguments)
+
-            user_changed = True
+        if group is None:
-        if self._module.check_mode or self._gitlab.addsshkeyuser(user_id=user_id, title=user_sshkey_name, key=user_sshkey_file):
+            return False
-            user_changed = True
+
-        if group_id != '':
+        if self.memberExists(group, self.getUserId(user)):
-            if self._module.check_mode or self.addToGroup(group_id, user_id, access_level):
+            member = self.findMember(group, self.getUserId(user))
-                user_changed = True
+            if not self.memberAsGoodAccessLevel(group, member.id, self.ACCESS_LEVEL[access_level]):
-        if user_changed:
+                member.access_level = self.ACCESS_LEVEL[access_level]
-            self._module.exit_json(changed=True, result="The user %s is updated" % user_username)
+                member.save()
+                return True
         else:
-            self._module.exit_json(changed=False, result="The user %s is already up2date" % user_username)
+            try:
+                group.members.create({
+                    'user_id': self.getUserId(user),
+                    'access_level': self.ACCESS_LEVEL[access_level]})
+            except gitlab.exceptions.GitlabCreateError as e:
+                self._module.fail_json(msg="Failed to assign user to group: %s" % to_native(e))
+            return True
+        return False
+
+    '''
+    @param user User object
+    @param arguments User attributes
+    '''
+    def updateUser(self, user, arguments):
+        changed = False
+
+        for arg_key, arg_value in arguments.items():
+            if arguments[arg_key] is not None:
+                if getattr(user, arg_key) != arguments[arg_key]:
+                    setattr(user, arg_key, arguments[arg_key])
+                    changed = True
+
+        return (changed, user)
+
+    '''
+    @param arguments User attributes
+    '''
+    def createUser(self, arguments):
+        if self._module.check_mode:
+            return True
+
+        try:
+            user = self._gitlab.users.create(arguments)
+        except (gitlab.exceptions.GitlabCreateError) as e:
+            self._module.fail_json(msg="Failed to create user: %s " % to_native(e))
+
+        return user
+
+    '''
+    @param username Username of the user
+    '''
+    def findUser(self, username):
+        users = self._gitlab.users.list(search=username)
+        for user in users:
+            if (user.username == username):
+                return user
+
+    '''
+    @param username Username of the user
+    '''
+    def existsUser(self, username):
+        # When user exists, object will be stored in self.userObject.
+        user = self.findUser(username)
+        if user:
+            self.userObject = user
+            return True
+        return False
+
+    def deleteUser(self):
+        if self._module.check_mode:
+            return True
+
+        user = self.userObject
+
+        return user.delete()
+
+
+def deprecation_warning(module):
+    deprecated_aliases = ['login_token']
+
+    module.deprecate("Aliases \'{aliases}\' are deprecated".format(aliases='\', \''.join(deprecated_aliases)), 2.10)
 
 
 def main():
+    argument_spec = basic_auth_argument_spec()
+    argument_spec.update(dict(
+        server_url=dict(type='str', required=True, removed_in_version=2.10),
+        login_user=dict(type='str', no_log=True, removed_in_version=2.10),
+        login_password=dict(type='str', no_log=True, removed_in_version=2.10),
+        api_token=dict(type='str', no_log=True, aliases=["login_token"]),
+        name=dict(type='str', required=True),
+        state=dict(type='str', default="present", choices=["absent", "present"]),
+        username=dict(type='str', required=True),
+        password=dict(type='str', required=True, no_log=True),
+        email=dict(type='str', required=True),
+        sshkey_name=dict(type='str'),
+        sshkey_file=dict(type='str'),
+        group=dict(type='str'),
+        access_level=dict(type='str', default="guest", choices=["developer", "guest", "maintainer", "master", "owner", "reporter"]),
+        confirm=dict(type='bool', default=True),
+        isadmin=dict(type='bool', default=False),
+        external=dict(type='bool', default=False),
+    ))
+
     module = AnsibleModule(
-        argument_spec=dict(
+        argument_spec=argument_spec,
-            server_url=dict(required=True),
+        mutually_exclusive=[
-            validate_certs=dict(required=False, default=True, type='bool', aliases=['verify_ssl']),
+            ['api_url', 'server_url'],
-            login_user=dict(required=False, no_log=True),
+            ['api_username', 'login_user'],
-            login_password=dict(required=False, no_log=True),
+            ['api_password', 'login_password'],
-            login_token=dict(required=False, no_log=True),
+            ['api_username', 'api_token'],
-            name=dict(required=True),
+            ['api_password', 'api_token'],
-            username=dict(required=True),
+            ['login_user', 'login_token'],
-            password=dict(required=True, no_log=True),
+            ['login_password', 'login_token']
-            email=dict(required=True),
+        ],
-            sshkey_name=dict(required=False),
+        required_together=[
-            sshkey_file=dict(required=False),
+            ['api_username', 'api_password'],
-            group=dict(required=False),
+            ['login_user', 'login_password'],
-            access_level=dict(required=False, choices=["guest", "reporter", "developer", "master", "owner"]),
+        ],
-            state=dict(default="present", choices=["present", "absent"]),
+        required_one_of=[
-            confirm=dict(required=False, default=True, type='bool')
+            ['api_username', 'api_token', 'login_user', 'login_token']
-        ),
+        ],
-        supports_check_mode=True
+        supports_check_mode=True,
     )
 
-    if not HAS_GITLAB_PACKAGE:
+    deprecation_warning(module)
-        module.fail_json(msg="Missing required gitlab module (check docs or install with: pip install pyapi-gitlab")
 
     server_url = module.params['server_url']
-    verify_ssl = module.params['validate_certs']
     login_user = module.params['login_user']
     login_password = module.params['login_password']
-    login_token = module.params['login_token']
+
+    api_url = module.params['api_url']
+    validate_certs = module.params['validate_certs']
+    api_user = module.params['api_username']
+    api_password = module.params['api_password']
+
+    gitlab_url = server_url if api_url is None else api_url
+    gitlab_user = login_user if api_user is None else api_user
+    gitlab_password = login_password if api_password is None else api_password
+    gitlab_token = module.params['api_token']
+
     user_name = module.params['name']
-    user_username = module.params['username']
+    state = module.params['state']
+    user_username = module.params['username'].lower()
     user_password = module.params['password']
     user_email = module.params['email']
     user_sshkey_name = module.params['sshkey_name']
     user_sshkey_file = module.params['sshkey_file']
-    group_name = module.params['group']
+    group_path = module.params['group']
     access_level = module.params['access_level']
-    state = module.params['state']
     confirm = module.params['confirm']
+    user_isadmin = module.params['isadmin']
+    user_external = module.params['external']
 
-    if len(user_password) < 8:
+    if not HAS_GITLAB_PACKAGE:
-        module.fail_json(msg="New user's 'password' should contain more than 8 characters.")
+        module.fail_json(msg=missing_required_lib("python-gitlab"), exception=GITLAB_IMP_ERR)
-
-    # We need both login_user and login_password or login_token, otherwise we fail.
-    if login_user is not None and login_password is not None:
-        use_credentials = True
-    elif login_token is not None:
-        use_credentials = False
-    else:
-        module.fail_json(msg="No login credentials are given. Use login_user with login_password, or login_token")
-
-    # Check if vars are none
-    if user_sshkey_file is not None and user_sshkey_name is not None:
-        use_sshkey = True
-    else:
-        use_sshkey = False
-
-    if group_name is not None and access_level is not None:
-        add_to_group = True
-        group_name = group_name.lower()
-    else:
-        add_to_group = False
-
-    user_username = user_username.lower()
 
-    # Lets make an connection to the Gitlab server_url, with either login_user and login_password
-    # or with login_token
     try:
-        if use_credentials:
+        gitlab_instance = gitlab.Gitlab(url=gitlab_url, ssl_verify=validate_certs, email=gitlab_user, password=gitlab_password,
-            git = gitlab.Gitlab(host=server_url, verify_ssl=verify_ssl)
+                                        private_token=gitlab_token, api_version=4)
-            git.login(user=login_user, password=login_password)
+        gitlab_instance.auth()
-        else:
+    except (gitlab.exceptions.GitlabAuthenticationError, gitlab.exceptions.GitlabGetError) as e:
-            git = gitlab.Gitlab(server_url, token=login_token, verify_ssl=verify_ssl)
-    except Exception as e:
         module.fail_json(msg="Failed to connect to Gitlab server: %s" % to_native(e))
+    except (gitlab.exceptions.GitlabHttpError) as e:
+        module.fail_json(msg="Failed to connect to Gitlab server: %s. \
+            Gitlab remove Session API now that private tokens are removed from user API endpoints since version 10.2." % to_native(e))
 
-    # Check if user is authorized or not before proceeding to any operations
+    gitlab_user = GitLabUser(module, gitlab_instance)
-    # if not, exit from here
+    user_exists = gitlab_user.existsUser(user_username)
-    auth_msg = git.currentuser().get('message', None)
-    if auth_msg is not None and auth_msg == '401 Unauthorized':
-        module.fail_json(msg='User unauthorized',
-                         details="User is not allowed to access Gitlab server "
-                                 "using login_token. Please check login_token")
-
-    # Validate if group exists and take action based on "state"
-    user = GitLabUser(module, git)
 
-    # Check if user exists, if not exists and state = absent, we exit nicely.
+    if state == 'absent':
-    if not user.existsUser(user_username) and state == "absent":
+        if user_exists:
-        module.exit_json(changed=False, result="User already deleted or does not exist")
+            gitlab_user.deleteUser()
+            module.exit_json(changed=True, msg="Successfully deleted user %s" % user_username)
         else:
-        # User exists,
+            module.exit_json(changed=False, msg="User deleted or does not exists")
-        if state == "absent":
+
-            user.deleteUser(user_username)
+    if state == 'present':
+        if gitlab_user.createOrUpdateUser(user_username, {
+                                          "name": user_name,
+                                          "password": user_password,
+                                          "email": user_email,
+                                          "sshkey_name": user_sshkey_name,
+                                          "sshkey_file": user_sshkey_file,
+                                          "group_path": group_path,
+                                          "access_level": access_level,
+                                          "confirm": confirm,
+                                          "isadmin": user_isadmin,
+                                          "external": user_external}):
+            module.exit_json(changed=True, msg="Successfully created or updated the user %s" % user_username, user=gitlab_user.userObject._attrs)
         else:
-            user.createOrUpdateUser(user_name, user_username, user_password, user_email, user_sshkey_name, user_sshkey_file, group_name, access_level, confirm)
+            module.exit_json(changed=False, msg="No need to update the user %s" % user_username, user=gitlab_user.userObject._attrs)
 
 
 if __name__ == '__main__':

test/runner/requirements/units.txt

@@ -38,3 +38,7 @@ pexpect
 # requirement for the linode module
 linode-python # APIv3
 linode_api4 ; python_version > '2.6' # APIv4
+
+# requirement for the gitlab module
+python-gitlab
+httmock

test/sanity/validate-modules/ignore.txt

@@ -822,9 +822,6 @@ lib/ansible/modules/remote_management/ucs/ucs_wwn_pool.py E322
 lib/ansible/modules/remote_management/ucs/ucs_wwn_pool.py E323
 lib/ansible/modules/source_control/github_issue.py E324
 lib/ansible/modules/source_control/github_issue.py E326
-lib/ansible/modules/source_control/gitlab_project.py E324
-lib/ansible/modules/source_control/gitlab_project.py E326
-lib/ansible/modules/source_control/gitlab_user.py E326
 lib/ansible/modules/source_control/subversion.py E322
 lib/ansible/modules/storage/infinidat/infini_export.py E323
 lib/ansible/modules/storage/infinidat/infini_export.py E324

test/units/modules/source_control/gitlab.py

@@ -0,0 +1,563 @@
+# -*- coding: utf-8 -*-
+
+# Copyright: (c) 2019, Guillaume Martinez (lunik@tiwabbit.fr)
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import absolute_import
+
+import sys
+
+from httmock import response  # noqa
+from httmock import urlmatch  # noqa
+
+from units.compat import unittest
+
+from gitlab import Gitlab
+
+
+class FakeAnsibleModule(object):
+    def __init__(self):
+        self.check_mode = False
+
+    def fail_json(self, **args):
+        pass
+
+    def exit_json(self, **args):
+        pass
+
+
+class GitlabModuleTestCase(unittest.TestCase):
+    def setUp(self):
+        unitest_python_version_check_requirement(self)
+
+        self.mock_module = FakeAnsibleModule()
+
+        self.gitlab_instance = Gitlab("http://localhost", private_token="private_token", api_version=4)
+
+
+# Python 2.7+ is needed for python-gitlab
+GITLAB_MINIMUM_PYTHON_VERSION = (2, 7)
+
+
+# Verify if the current Python version is higher than GITLAB_MINIMUM_PYTHON_VERSION
+def python_version_match_requirement():
+    return sys.version_info >= GITLAB_MINIMUM_PYTHON_VERSION
+
+
+# Skip unittest test case if python version don't match requirement
+def unitest_python_version_check_requirement(unittest_testcase):
+    if not python_version_match_requirement():
+        unittest_testcase.skipTest("Python %s+ is needed for python-gitlab" % ",".join(map(str, GITLAB_MINIMUM_PYTHON_VERSION)))
+
+
+'''
+USER API
+'''
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/users", method="get")
+def resp_find_user(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('[{"id": 1, "username": "john_smith", "name": "John Smith", "state": "active",'
+               '"avatar_url": "http://localhost:3000/uploads/user/avatar/1/cd8.jpeg",'
+               '"web_url": "http://localhost:3000/john_smith"}, {"id": 2,'
+               '"username": "jack_smith", "name": "Jack Smith", "state": "blocked",'
+               '"avatar_url": "http://gravatar.com/../e32131cd8.jpeg",'
+               '"web_url": "http://localhost:3000/jack_smith"}]')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/users", method="post")
+def resp_create_user(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"id": 1, "username": "john_smith", "name": "John Smith", "state": "active",'
+               '"avatar_url": "http://localhost:3000/uploads/user/avatar/1/cd8.jpeg",'
+               '"web_url": "http://localhost:3000/john_smith","created_at": "2012-05-23T08:00:58Z",'
+               '"bio": null, "location": null, "public_email": "john@example.com", "skype": "",'
+               '"linkedin": "", "twitter": "", "website_url": "", "organization": ""}')
+    content = content.encode("utf-8")
+    return response(201, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/users/1", method="get")
+def resp_get_user(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"id": 1, "username": "john_smith", "name": "John Smith",'
+               '"state": "active",'
+               '"avatar_url": "http://localhost:3000/uploads/user/avatar/1/cd8.jpeg",'
+               '"web_url": "http://localhost:3000/john_smith",'
+               '"created_at": "2012-05-23T08:00:58Z", "bio": null, "location": null,'
+               '"public_email": "john@example.com", "skype": "", "linkedin": "",'
+               '"twitter": "", "website_url": "", "organization": "", "is_admin": false}')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/users/1", method="get")
+def resp_get_missing_user(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{}')
+    content = content.encode("utf-8")
+    return response(404, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/users/1", method="delete")
+def resp_delete_user(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{}')
+    content = content.encode("utf-8")
+    return response(204, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/users/1", method="delete")
+def resp_delete_missing_user(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{}')
+    content = content.encode("utf-8")
+    return response(404, content, headers, None, 5, request)
+
+
+'''
+USER SSHKEY API
+'''
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/users/1/keys", method="get")
+def resp_get_user_keys(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('[{"id": 1, "title": "Public key",'
+               '"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAiPWx6WM4lhHNedGfBpPJNPpZ7yKu+dnn1SJejgt4596'
+               'k6YjzGGphH2TUxwKzxcKDKKezwkpfnxPkSMkuEspGRt/aZZ9wa++Oi7Qkr8prgHc4soW6NUlfDzpvZK2H5E7eQa'
+               'SeP3SAwGmQKUFHCddNaP0L+hM7zhFNzjFvpaMgJw0=",'
+               '"created_at": "2014-08-01T14:47:39.080Z"},{"id": 3,'
+               '"title": "Another Public key",'
+               '"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAiPWx6WM4lhHNedGfBpPJNPpZ7yKu+dnn1SJejgt4596'
+               'k6YjzGGphH2TUxwKzxcKDKKezwkpfnxPkSMkuEspGRt/aZZ9wa++Oi7Qkr8prgHc4soW6NUlfDzpvZK2H5E7eQaS'
+               'eP3SAwGmQKUFHCddNaP0L+hM7zhFNzjFvpaMgJw0=",'
+               '"created_at": "2014-08-01T14:47:39.080Z"}]')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/users/1/keys", method="post")
+def resp_create_user_keys(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"id": 1, "title": "Private key",'
+               '"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDA1YotVDm2mAyk2tPt4E7AHm01sS6JZmcUdRuSuA5z'
+               'szUJzYPPUSRAX3BCgTqLqYx//UuVncK7YqLVSbbwjKR2Ez5lISgCnVfLVEXzwhv+xawxKWmI7hJ5S0tOv6MJ+Ixy'
+               'Ta4xcKwJTwB86z22n9fVOQeJTR2dSOH1WJrf0PvRk+KVNY2jTiGHTi9AIjLnyD/jWRpOgtdfkLRc8EzAWrWlgNmH'
+               '2WOKBw6za0az6XoG75obUdFVdW3qcD0xc809OHLi7FDf+E7U4wiZJCFuUizMeXyuK/SkaE1aee4Qp5R4dxTR4TP9'
+               'M1XAYkf+kF0W9srZ+mhF069XD/zhUPJsvwEF",'
+               '"created_at": "2014-08-01T14:47:39.080Z"}')
+    content = content.encode("utf-8")
+    return response(201, content, headers, None, 5, request)
+
+
+'''
+GROUP API
+'''
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/groups", method="get")
+def resp_find_group(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('[{"id": 1, "name": "Foobar Group", "path": "foo-bar",'
+               '"description": "An interesting group", "visibility": "public",'
+               '"lfs_enabled": true, "avatar_url": "http://localhost:3000/uploads/group/avatar/1/foo.jpg",'
+               '"web_url": "http://localhost:3000/groups/foo-bar", "request_access_enabled": false,'
+               '"full_name": "Foobar Group", "full_path": "foo-bar",'
+               '"file_template_project_id": 1, "parent_id": null, "projects": []}, {"id": 2, "name": "BarFoo Group", "path": "bar-foor",'
+               '"description": "An interesting group", "visibility": "public",'
+               '"lfs_enabled": true, "avatar_url": "http://localhost:3000/uploads/group/avatar/2/bar.jpg",'
+               '"web_url": "http://localhost:3000/groups/bar-foo", "request_access_enabled": false,'
+               '"full_name": "BarFoo Group", "full_path": "bar-foo",'
+               '"file_template_project_id": 1, "parent_id": null, "projects": []}]')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/groups/1", method="get")
+def resp_get_group(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"id": 1, "name": "Foobar Group", "path": "foo-bar",'
+               '"description": "An interesting group", "visibility": "public",'
+               '"lfs_enabled": true, "avatar_url": "http://localhost:3000/uploads/group/avatar/1/foo.jpg",'
+               '"web_url": "http://localhost:3000/groups/foo-bar", "request_access_enabled": false,'
+               '"full_name": "Foobar Group", "full_path": "foo-bar",'
+               '"file_template_project_id": 1, "parent_id": null, "projects": [{"id": 1,"description": null, "default_branch": "master",'
+               '"ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git",'
+               '"http_url_to_repo": "http://example.com/diaspora/diaspora-client.git",'
+               '"web_url": "http://example.com/diaspora/diaspora-client",'
+               '"readme_url": "http://example.com/diaspora/diaspora-client/blob/master/README.md",'
+               '"tag_list": ["example","disapora client"],"name": "Diaspora Client",'
+               '"name_with_namespace": "Diaspora / Diaspora Client","path": "diaspora-client",'
+               '"path_with_namespace": "diaspora/diaspora-client","created_at": "2013-09-30T13:46:02Z",'
+               '"last_activity_at": "2013-09-30T13:46:02Z","forks_count": 0,'
+               '"avatar_url": "http://example.com/uploads/project/avatar/4/uploads/avatar.png",'
+               '"star_count": 0}]}')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/groups/1", method="get")
+def resp_get_missing_group(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{}')
+    content = content.encode("utf-8")
+    return response(404, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/groups", method="post")
+def resp_create_group(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"id": 1, "name": "Foobar Group", "path": "foo-bar",'
+               '"description": "An interesting group", "visibility": "public",'
+               '"lfs_enabled": true, "avatar_url": "http://localhost:3000/uploads/group/avatar/1/foo.jpg",'
+               '"web_url": "http://localhost:3000/groups/foo-bar", "request_access_enabled": false,'
+               '"full_name": "Foobar Group", "full_path": "foo-bar",'
+               '"file_template_project_id": 1, "parent_id": null}')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/groups", method="post")
+def resp_create_subgroup(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"id": 2, "name": "BarFoo Group", "path": "bar-foor",'
+               '"description": "An interesting group", "visibility": "public",'
+               '"lfs_enabled": true, "avatar_url": "http://localhost:3000/uploads/group/avatar/2/bar.jpg",'
+               '"web_url": "http://localhost:3000/groups/foo-bar/bar-foo", "request_access_enabled": false,'
+               '"full_name": "BarFoo Group", "full_path": "foo-bar/bar-foo",'
+               '"file_template_project_id": 1, "parent_id": 1}')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/users/1", method="delete")
+def resp_delete_group(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{}')
+    content = content.encode("utf-8")
+    return response(204, content, headers, None, 5, request)
+
+
+'''
+GROUP MEMBER API
+'''
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/groups/1/members/1", method="get")
+def resp_get_member(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"id": 1, "username": "raymond_smith", "name": "Raymond Smith", "state": "active",'
+               '"avatar_url": "https://www.gravatar.com/avatar/c2525a7f58ae3776070e44c106c48e15?s=80&d=identicon",'
+               '"web_url": "http://192.168.1.8:3000/root", "expires_at": "2012-10-22T14:13:35Z", "access_level": 30}')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/groups/1/members", method="get")
+def resp_find_member(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('[{"id": 1, "username": "raymond_smith", "name": "Raymond Smith", "state": "active",'
+               '"avatar_url": "https://www.gravatar.com/avatar/c2525a7f58ae3776070e44c106c48e15?s=80&d=identicon",'
+               '"web_url": "http://192.168.1.8:3000/root", "expires_at": "2012-10-22T14:13:35Z", "access_level": 30},{'
+               '"id": 2, "username": "john_doe", "name": "John Doe","state": "active",'
+               '"avatar_url": "https://www.gravatar.com/avatar/c2525a7f58ae3776070e44c106c48e15?s=80&d=identicon",'
+               '"web_url": "http://192.168.1.8:3000/root","expires_at": "2012-10-22T14:13:35Z",'
+               '"access_level": 30}]')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/groups/1/members", method="post")
+def resp_add_member(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"id": 1, "username": "raymond_smith", "name": "Raymond Smith",'
+               '"state": "active",'
+               '"avatar_url": "https://www.gravatar.com/avatar/c2525a7f58ae3776070e44c106c48e15?s=80&d=identicon",'
+               '"web_url": "http://192.168.1.8:3000/root", "expires_at": "2012-10-22T14:13:35Z",'
+               '"access_level": 30}')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/groups/1/members/1", method="put")
+def resp_update_member(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"id": 1, "username": "raymond_smith", "name": "Raymond Smith",'
+               '"state": "active",'
+               '"avatar_url": "https://www.gravatar.com/avatar/c2525a7f58ae3776070e44c106c48e15?s=80&d=identicon",'
+               '"web_url": "http://192.168.1.8:3000/root", "expires_at": "2012-10-22T14:13:35Z",'
+               '"access_level": 10}')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+'''
+DEPLOY KEY API
+'''
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/projects/1/deploy_keys", method="get")
+def resp_find_project_deploy_key(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('[{"id": 1,"title": "Public key",'
+               '"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAiPWx6WM4lhHNedGfBpPJNPpZ7yKu+dnn1SJejgt4596k6YjzGGphH2TUxwKzxc'
+               'KDKKezwkpfnxPkSMkuEspGRt/aZZ9wa++Oi7Qkr8prgHc4soW6NUlfDzpvZK2H5E7eQaSeP3SAwGmQKUFHCddNaP0L+hM7zhFNzjFvpaMgJw0=",'
+               '"created_at": "2013-10-02T10:12:29Z"},{"id": 3,"title": "Another Public key",'
+               '"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAiPWx6WM4lhHNedGfBpPJNPpZ7yKu+dnn1SJejgt4596k6YjzGGphH2TUxwKzxc'
+               'KDKKezwkpfnxPkSMkuEspGRt/aZZ9wa++Oi7Qkr8prgHc4soW6NUlfDzpvZK2H5E7eQaSeP3SAwGmQKUFHCddNaP0L+hM7zhFNzjFvpaMgJw0=",'
+               '"created_at": "2013-10-02T11:12:29Z"}]')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/projects/1/deploy_keys/1", method="get")
+def resp_get_project_deploy_key(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"id": 1,"title": "Public key",'
+               '"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAiPWx6WM4lhHNedGfBpPJNPpZ7yKu+dnn1SJejgt4596k6YjzGGphH2TUxwKzxc'
+               'KDKKezwkpfnxPkSMkuEspGRt/aZZ9wa++Oi7Qkr8prgHc4soW6NUlfDzpvZK2H5E7eQaSeP3SAwGmQKUFHCddNaP0L+hM7zhFNzjFvpaMgJw0=",'
+               '"created_at": "2013-10-02T10:12:29Z"}')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/projects/1/deploy_keys", method="post")
+def resp_create_project_deploy_key(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"id": 1,"title": "Public key",'
+               '"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAiPWx6WM4lhHNedGfBpPJNPpZ7yKu+dnn1SJejgt4596k6YjzGGphH2TUxwKzxc'
+               'KDKKezwkpfnxPkSMkuEspGRt/aZZ9wa++Oi7Qkr8prgHc4soW6NUlfDzpvZK2H5E7eQaSeP3SAwGmQKUFHCddNaP0L+hM7zhFNzjFvpaMgJw0=",'
+               '"created_at": "2013-10-02T10:12:29Z"}')
+    content = content.encode("utf-8")
+    return response(201, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/projects/1/deploy_keys/1", method="delete")
+def resp_delete_project_deploy_key(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{}')
+    content = content.encode("utf-8")
+    return response(204, content, headers, None, 5, request)
+
+
+'''
+PROJECT API
+'''
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/projects", method="get")
+def resp_find_project(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('[{"id": 1,"description": null, "default_branch": "master",'
+               '"ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git",'
+               '"http_url_to_repo": "http://example.com/diaspora/diaspora-client.git",'
+               '"web_url": "http://example.com/diaspora/diaspora-client",'
+               '"readme_url": "http://example.com/diaspora/diaspora-client/blob/master/README.md",'
+               '"tag_list": ["example","disapora client"],"name": "Diaspora Client",'
+               '"name_with_namespace": "Diaspora / Diaspora Client","path": "diaspora-client",'
+               '"path_with_namespace": "diaspora/diaspora-client","created_at": "2013-09-30T13:46:02Z",'
+               '"last_activity_at": "2013-09-30T13:46:02Z","forks_count": 0,'
+               '"avatar_url": "http://example.com/uploads/project/avatar/4/uploads/avatar.png",'
+               '"star_count": 0}]')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/projects/1", method="get")
+def resp_get_project(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"id": 1,"description": null, "default_branch": "master",'
+               '"ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git",'
+               '"http_url_to_repo": "http://example.com/diaspora/diaspora-client.git",'
+               '"web_url": "http://example.com/diaspora/diaspora-client",'
+               '"readme_url": "http://example.com/diaspora/diaspora-client/blob/master/README.md",'
+               '"tag_list": ["example","disapora client"],"name": "Diaspora Client",'
+               '"name_with_namespace": "Diaspora / Diaspora Client","path": "diaspora-client",'
+               '"path_with_namespace": "diaspora/diaspora-client","created_at": "2013-09-30T13:46:02Z",'
+               '"last_activity_at": "2013-09-30T13:46:02Z","forks_count": 0,'
+               '"avatar_url": "http://example.com/uploads/project/avatar/4/uploads/avatar.png",'
+               '"star_count": 0}')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/projects/foo-bar%2Fdiaspora-client", method="get")
+def resp_get_project_by_name(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"id": 1,"description": null, "default_branch": "master",'
+               '"ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git",'
+               '"http_url_to_repo": "http://example.com/diaspora/diaspora-client.git",'
+               '"web_url": "http://example.com/diaspora/diaspora-client",'
+               '"readme_url": "http://example.com/diaspora/diaspora-client/blob/master/README.md",'
+               '"tag_list": ["example","disapora client"],"name": "Diaspora Client",'
+               '"name_with_namespace": "Diaspora / Diaspora Client","path": "diaspora-client",'
+               '"path_with_namespace": "diaspora/diaspora-client","created_at": "2013-09-30T13:46:02Z",'
+               '"last_activity_at": "2013-09-30T13:46:02Z","forks_count": 0,'
+               '"avatar_url": "http://example.com/uploads/project/avatar/4/uploads/avatar.png",'
+               '"star_count": 0}')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/groups/1/projects", method="get")
+def resp_find_group_project(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('[{"id": 1,"description": null, "default_branch": "master",'
+               '"ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git",'
+               '"http_url_to_repo": "http://example.com/diaspora/diaspora-client.git",'
+               '"web_url": "http://example.com/diaspora/diaspora-client",'
+               '"readme_url": "http://example.com/diaspora/diaspora-client/blob/master/README.md",'
+               '"tag_list": ["example","disapora client"],"name": "Diaspora Client",'
+               '"name_with_namespace": "Diaspora / Diaspora Client","path": "diaspora-client",'
+               '"path_with_namespace": "diaspora/diaspora-client","created_at": "2013-09-30T13:46:02Z",'
+               '"last_activity_at": "2013-09-30T13:46:02Z","forks_count": 0,'
+               '"avatar_url": "http://example.com/uploads/project/avatar/4/uploads/avatar.png",'
+               '"star_count": 0}]')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/groups/1/projects/1", method="get")
+def resp_get_group_project(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"id": 1,"description": null, "default_branch": "master",'
+               '"ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git",'
+               '"http_url_to_repo": "http://example.com/diaspora/diaspora-client.git",'
+               '"web_url": "http://example.com/diaspora/diaspora-client",'
+               '"readme_url": "http://example.com/diaspora/diaspora-client/blob/master/README.md",'
+               '"tag_list": ["example","disapora client"],"name": "Diaspora Client",'
+               '"name_with_namespace": "Diaspora / Diaspora Client","path": "diaspora-client",'
+               '"path_with_namespace": "diaspora/diaspora-client","created_at": "2013-09-30T13:46:02Z",'
+               '"last_activity_at": "2013-09-30T13:46:02Z","forks_count": 0,'
+               '"avatar_url": "http://example.com/uploads/project/avatar/4/uploads/avatar.png",'
+               '"star_count": 0}')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/projects", method="post")
+def resp_create_project(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"id": 1,"description": null, "default_branch": "master",'
+               '"ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git",'
+               '"http_url_to_repo": "http://example.com/diaspora/diaspora-client.git",'
+               '"web_url": "http://example.com/diaspora/diaspora-client",'
+               '"readme_url": "http://example.com/diaspora/diaspora-client/blob/master/README.md",'
+               '"tag_list": ["example","disapora client"],"name": "Diaspora Client",'
+               '"name_with_namespace": "Diaspora / Diaspora Client","path": "diaspora-client",'
+               '"path_with_namespace": "diaspora/diaspora-client","created_at": "2013-09-30T13:46:02Z",'
+               '"last_activity_at": "2013-09-30T13:46:02Z","forks_count": 0,'
+               '"avatar_url": "http://example.com/uploads/project/avatar/4/uploads/avatar.png",'
+               '"star_count": 0}')
+    content = content.encode("utf-8")
+    return response(201, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/projects/1", method="delete")
+def resp_delete_project(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{}')
+    content = content.encode("utf-8")
+
+    return response(204, content, headers, None, 5, request)
+
+
+'''
+HOOK API
+'''
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/projects/1/hooks", method="get")
+def resp_find_project_hook(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('[{"id": 1,"url": "http://example.com/hook","project_id": 3,'
+               '"push_events": true,"push_events_branch_filter": "","issues_events": true,'
+               '"confidential_issues_events": true,"merge_requests_events": true,'
+               '"tag_push_events": true,"note_events": true,"job_events": true,'
+               '"pipeline_events": true,"wiki_page_events": true,"enable_ssl_verification": true,'
+               '"created_at": "2012-10-12T17:04:47Z"}]')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/projects/1/hooks/1", method="get")
+def resp_get_project_hook(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"id": 1,"url": "http://example.com/hook","project_id": 3,'
+               '"push_events": true,"push_events_branch_filter": "","issues_events": true,'
+               '"confidential_issues_events": true,"merge_requests_events": true,'
+               '"tag_push_events": true,"note_events": true,"job_events": true,'
+               '"pipeline_events": true,"wiki_page_events": true,"enable_ssl_verification": true,'
+               '"created_at": "2012-10-12T17:04:47Z"}')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/projects/1/hooks", method="post")
+def resp_create_project_hook(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"id": 1,"url": "http://example.com/hook","project_id": 3,'
+               '"push_events": true,"push_events_branch_filter": "","issues_events": true,'
+               '"confidential_issues_events": true,"merge_requests_events": true,'
+               '"tag_push_events": true,"note_events": true,"job_events": true,'
+               '"pipeline_events": true,"wiki_page_events": true,"enable_ssl_verification": true,'
+               '"created_at": "2012-10-12T17:04:47Z"}')
+    content = content.encode("utf-8")
+    return response(201, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/projects/1/hooks/1", method="delete")
+def resp_delete_project_hook(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{}')
+    content = content.encode("utf-8")
+    return response(204, content, headers, None, 5, request)
+
+
+'''
+HOOK API
+'''
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/runners/all", method="get")
+def resp_find_runners(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('[{"active": true,"description": "test-1-20150125","id": 1,'
+               '"is_shared": false,"ip_address": "127.0.0.1","name": null,'
+               '"online": true,"status": "online"},{"active": true,'
+               '"description": "test-2-20150125","id": 2,"ip_address": "127.0.0.1",'
+               '"is_shared": false,"name": null,"online": false,"status": "offline"}]')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/runners/1", method="get")
+def resp_get_runner(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"active": true,"description": "test-1-20150125","id": 1,'
+               '"is_shared": false,"ip_address": "127.0.0.1","name": null,'
+               '"online": true,"status": "online"}')
+    content = content.encode("utf-8")
+    return response(200, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/runners", method="post")
+def resp_create_runner(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{"active": true,"description": "test-1-20150125","id": 1,'
+               '"is_shared": false,"ip_address": "127.0.0.1","name": null,'
+               '"online": true,"status": "online"}')
+    content = content.encode("utf-8")
+    return response(201, content, headers, None, 5, request)
+
+
+@urlmatch(scheme="http", netloc="localhost", path="/api/v4/runners/1", method="delete")
+def resp_delete_runner(url, request):
+    headers = {'content-type': 'application/json'}
+    content = ('{}')
+    content = content.encode("utf-8")
+    return response(204, content, headers, None, 5, request)

test/units/modules/source_control/test_gitlab_deploy_key.py

@@ -1,233 +1,84 @@
 # -*- coding: utf-8 -*-
-# Copyright (c) 2018 Marcus Watkins <marwatk@marcuswatkins.net>
+
+# Copyright: (c) 2019, Guillaume Martinez (lunik@tiwabbit.fr)
 # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
 
-from units.compat.mock import patch
+from __future__ import absolute_import
-from ansible.modules.source_control import gitlab_deploy_key
+
-from ansible.module_utils._text import to_bytes
+from ansible.modules.source_control.gitlab_deploy_key import GitLabDeployKey
-from ansible.module_utils import basic
+
+from .gitlab import (GitlabModuleTestCase,
+                     python_version_match_requirement,
+                     resp_get_project, resp_find_project_deploy_key,
+                     resp_create_project_deploy_key, resp_delete_project_deploy_key)
+
+# Gitlab module requirements
+if python_version_match_requirement():
+    from gitlab.v4.objects import ProjectKey
+
+# Unit tests requirements
+from httmock import with_httmock  # noqa
+
+
+class TestGitlabDeployKey(GitlabModuleTestCase):
+    def setUp(self):
+        super(TestGitlabDeployKey, self).setUp()
+
+        self.moduleUtil = GitLabDeployKey(module=self.mock_module, gitlab_instance=self.gitlab_instance)
+
+    @with_httmock(resp_get_project)
+    @with_httmock(resp_find_project_deploy_key)
+    def test_deploy_key_exist(self):
+        project = self.gitlab_instance.projects.get(1)
+
+        rvalue = self.moduleUtil.existsDeployKey(project, "Public key")
+
+        self.assertEqual(rvalue, True)
+
+        rvalue = self.moduleUtil.existsDeployKey(project, "Private key")
+
+        self.assertEqual(rvalue, False)
+
+    @with_httmock(resp_get_project)
+    @with_httmock(resp_create_project_deploy_key)
+    def test_create_deploy_key(self):
+        project = self.gitlab_instance.projects.get(1)
+
+        deploy_key = self.moduleUtil.createDeployKey(project, {"title": "Public key",
+                                                               "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAiPWx6WM"
+                                                               "4lhHNedGfBpPJNPpZ7yKu+dnn1SJejgt4596k6YjzGGphH2TUxwKzxc"
+                                                               "KDKKezwkpfnxPkSMkuEspGRt/aZZ9wa++Oi7Qkr8prgHc4soW6NUlfD"
+                                                               "zpvZK2H5E7eQaSeP3SAwGmQKUFHCddNaP0L+hM7zhFNzjFvpaMgJw0="})
+
+        self.assertEqual(type(deploy_key), ProjectKey)
+        self.assertEqual(deploy_key.title, "Public key")
+
+    @with_httmock(resp_get_project)
+    @with_httmock(resp_find_project_deploy_key)
+    @with_httmock(resp_create_project_deploy_key)
+    def test_update_deploy_key(self):
+        project = self.gitlab_instance.projects.get(1)
+        deployKey = self.moduleUtil.findDeployKey(project, "Public key")
+
+        changed, newDeploy_key = self.moduleUtil.updateDeployKey(deployKey, {"title": "Private key"})
+
+        self.assertEqual(changed, True)
+        self.assertEqual(type(newDeploy_key), ProjectKey)
+        self.assertEqual(newDeploy_key.title, "Private key")
 
-import pytest
+        changed, newDeploy_key = self.moduleUtil.updateDeployKey(deployKey, {"title": "Private key"})
-import json
 
-from units.modules.utils import set_module_args
+        self.assertEqual(changed, False)
+        self.assertEqual(newDeploy_key.title, "Private key")
 
+    @with_httmock(resp_get_project)
+    @with_httmock(resp_find_project_deploy_key)
+    @with_httmock(resp_delete_project_deploy_key)
+    def test_delete_deploy_key(self):
+        project = self.gitlab_instance.projects.get(1)
 
-fake_server_state = [
+        self.moduleUtil.existsDeployKey(project, "Public key")
-    {
-        "id": 1,
-        "title": "Public key",
-        "key": 'ssh-rsa long/+base64//+string==',
-        "created_at": "2013-10-02T10:12:29Z",
-        "can_push": False
-    },
-]
 
+        rvalue = self.moduleUtil.deleteDeployKey()
 
-class FakeReader:
+        self.assertEqual(rvalue, None)
-    def __init__(self, object):
-        self.content = json.dumps(object, sort_keys=True)
-
-    def read(self):
-        return self.content
-
-
-class AnsibleExitJson(Exception):
-    """Exception class to be raised by module.exit_json and caught by the test case"""
-    pass
-
-
-class AnsibleFailJson(Exception):
-    """Exception class to be raised by module.fail_json and caught by the test case"""
-    pass
-
-
-def exit_json(*args, **kwargs):
-    """function to patch over exit_json; package return data into an exception"""
-    if 'changed' not in kwargs:
-        kwargs['changed'] = False
-    raise AnsibleExitJson(kwargs)
-
-
-def fail_json(*args, **kwargs):
-    """function to patch over fail_json; package return data into an exception"""
-    kwargs['failed'] = True
-    raise AnsibleFailJson(kwargs)
-
-
-@pytest.fixture
-def fetch_url_mock(mocker):
-    return mocker.patch('ansible.module_utils.gitlab.fetch_url')
-
-
-@pytest.fixture
-def module_mock(mocker):
-    return mocker.patch.multiple(basic.AnsibleModule,
-                                 exit_json=exit_json,
-                                 fail_json=fail_json)
-
-
-def test_access_token_output(capfd, fetch_url_mock, module_mock):
-    fetch_url_mock.return_value = [FakeReader(fake_server_state), {'status': 200}]
-    set_module_args({
-        'api_url': 'https://gitlab.example.com/api',
-        'access_token': 'test-access-token',
-        'project': '10',
-        'key': 'ssh-key foobar',
-        'title': 'a title',
-        'state': 'absent'
-    })
-    with pytest.raises(AnsibleExitJson) as result:
-        gitlab_deploy_key.main()
-
-    first_call = fetch_url_mock.call_args_list[0][1]
-    assert first_call['url'] == 'https://gitlab.example.com/api/v4/projects/10/deploy_keys'
-    assert first_call['headers']['Authorization'] == 'Bearer test-access-token'
-    assert 'Private-Token' not in first_call['headers']
-    assert first_call['method'] == 'GET'
-
-
-def test_private_token_output(capfd, fetch_url_mock, module_mock):
-    fetch_url_mock.return_value = [FakeReader(fake_server_state), {'status': 200}]
-    set_module_args({
-        'api_url': 'https://gitlab.example.com/api',
-        'private_token': 'test-private-token',
-        'project': 'foo/bar',
-        'key': 'ssh-key foobar',
-        'title': 'a title',
-        'state': 'absent'
-    })
-    with pytest.raises(AnsibleExitJson) as result:
-        gitlab_deploy_key.main()
-
-    first_call = fetch_url_mock.call_args_list[0][1]
-    assert first_call['url'] == 'https://gitlab.example.com/api/v4/projects/foo%2Fbar/deploy_keys'
-    assert first_call['headers']['Private-Token'] == 'test-private-token'
-    assert 'Authorization' not in first_call['headers']
-    assert first_call['method'] == 'GET'
-
-
-def test_bad_http_first_response(capfd, fetch_url_mock, module_mock):
-    fetch_url_mock.side_effect = [[FakeReader("Permission denied"), {'status': 403}], [FakeReader("Permission denied"), {'status': 403}]]
-    set_module_args({
-        'api_url': 'https://gitlab.example.com/api',
-        'access_token': 'test-access-token',
-        'project': '10',
-        'key': 'ssh-key foobar',
-        'title': 'a title',
-        'state': 'absent'
-    })
-    with pytest.raises(AnsibleFailJson):
-        gitlab_deploy_key.main()
-
-
-def test_bad_http_second_response(capfd, fetch_url_mock, module_mock):
-    fetch_url_mock.side_effect = [[FakeReader(fake_server_state), {'status': 200}], [FakeReader("Permission denied"), {'status': 403}]]
-    set_module_args({
-        'api_url': 'https://gitlab.example.com/api',
-        'access_token': 'test-access-token',
-        'project': '10',
-        'key': 'ssh-key foobar',
-        'title': 'a title',
-        'state': 'present'
-    })
-    with pytest.raises(AnsibleFailJson):
-        gitlab_deploy_key.main()
-
-
-def test_delete_non_existing(capfd, fetch_url_mock, module_mock):
-    fetch_url_mock.return_value = [FakeReader(fake_server_state), {'status': 200}]
-    set_module_args({
-        'api_url': 'https://gitlab.example.com/api',
-        'access_token': 'test-access-token',
-        'project': '10',
-        'key': 'ssh-key foobar',
-        'title': 'a title',
-        'state': 'absent'
-    })
-    with pytest.raises(AnsibleExitJson) as result:
-        gitlab_deploy_key.main()
-
-    assert result.value.args[0]['changed'] is False
-
-
-def test_delete_existing(capfd, fetch_url_mock, module_mock):
-    fetch_url_mock.return_value = [FakeReader(fake_server_state), {'status': 200}]
-    set_module_args({
-        'api_url': 'https://gitlab.example.com/api',
-        'access_token': 'test-access-token',
-        'project': '10',
-        'key': 'ssh-rsa long/+base64//+string==',
-        'title': 'a title',
-        'state': 'absent'
-    })
-    with pytest.raises(AnsibleExitJson) as result:
-        gitlab_deploy_key.main()
-
-    second_call = fetch_url_mock.call_args_list[1][1]
-
-    assert second_call['url'] == 'https://gitlab.example.com/api/v4/projects/10/deploy_keys/1'
-    assert second_call['method'] == 'DELETE'
-
-    assert result.value.args[0]['changed'] is True
-
-
-def test_add_new(capfd, fetch_url_mock, module_mock):
-    fetch_url_mock.return_value = [FakeReader(fake_server_state), {'status': 200}]
-    set_module_args({
-        'api_url': 'https://gitlab.example.com/api',
-        'access_token': 'test-access-token',
-        'project': '10',
-        'key': 'ssh-key foobar',
-        'title': 'a title',
-        'state': 'present'
-    })
-    with pytest.raises(AnsibleExitJson) as result:
-        gitlab_deploy_key.main()
-
-    second_call = fetch_url_mock.call_args_list[1][1]
-
-    assert second_call['url'] == 'https://gitlab.example.com/api/v4/projects/10/deploy_keys'
-    assert second_call['method'] == 'POST'
-    assert second_call['data'] == '{"can_push": false, "key": "ssh-key foobar", "title": "a title"}'
-    assert result.value.args[0]['changed'] is True
-
-
-def test_update_existing(capfd, fetch_url_mock, module_mock):
-    fetch_url_mock.return_value = [FakeReader(fake_server_state), {'status': 200}]
-    set_module_args({
-        'api_url': 'https://gitlab.example.com/api',
-        'access_token': 'test-access-token',
-        'project': '10',
-        'title': 'Public key',
-        'key': 'ssh-rsa long/+base64//+string==',
-        'can_push': 'yes',
-        'state': 'present'
-    })
-    with pytest.raises(AnsibleExitJson) as result:
-        gitlab_deploy_key.main()
-
-    second_call = fetch_url_mock.call_args_list[1][1]
-
-    assert second_call['url'] == 'https://gitlab.example.com/api/v4/projects/10/deploy_keys/1'
-    assert second_call['method'] == 'PUT'
-    assert second_call['data'] == ('{"can_push": true, "key": "ssh-rsa long/+base64//+string==", "title": "Public key"}')
-    assert result.value.args[0]['changed'] is True
-
-
-def test_unchanged_existing(capfd, fetch_url_mock, module_mock):
-    fetch_url_mock.return_value = [FakeReader(fake_server_state), {'status': 200}]
-    set_module_args({
-        'api_url': 'https://gitlab.example.com/api',
-        'access_token': 'test-access-token',
-        'project': '10',
-        'title': 'Public key',
-        'key': 'ssh-rsa long/+base64//+string==',
-        'can_push': 'no',
-        'state': 'present'
-    })
-    with pytest.raises(AnsibleExitJson) as result:
-        gitlab_deploy_key.main()
-
-    assert result.value.args[0]['changed'] is False
-    assert fetch_url_mock.call_count == 1

test/units/modules/source_control/test_gitlab_group.py

@@ -0,0 +1,83 @@
+# -*- coding: utf-8 -*-
+
+# Copyright: (c) 2019, Guillaume Martinez (lunik@tiwabbit.fr)
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import absolute_import
+
+from ansible.modules.source_control.gitlab_group import GitLabGroup
+
+from .gitlab import (GitlabModuleTestCase,
+                     python_version_match_requirement,
+                     resp_get_group, resp_get_missing_group, resp_create_group,
+                     resp_create_subgroup, resp_delete_group, resp_find_group_project)
+
+# Gitlab module requirements
+if python_version_match_requirement():
+    from gitlab.v4.objects import Group
+
+# Unit tests requirements
+from httmock import with_httmock  # noqa
+
+
+class TestGitlabGroup(GitlabModuleTestCase):
+    def setUp(self):
+        super(TestGitlabGroup, self).setUp()
+
+        self.moduleUtil = GitLabGroup(module=self.mock_module, gitlab_instance=self.gitlab_instance)
+
+    @with_httmock(resp_get_group)
+    def test_exist_group(self):
+        rvalue = self.moduleUtil.existsGroup(1)
+
+        self.assertEqual(rvalue, True)
+
+    @with_httmock(resp_get_missing_group)
+    def test_exist_group(self):
+        rvalue = self.moduleUtil.existsGroup(1)
+
+        self.assertEqual(rvalue, False)
+
+    @with_httmock(resp_create_group)
+    def test_create_group(self):
+        group = self.moduleUtil.createGroup({'name': "Foobar Group", 'path': "foo-bar"})
+
+        self.assertEqual(type(group), Group)
+        self.assertEqual(group.name, "Foobar Group")
+        self.assertEqual(group.path, "foo-bar")
+        self.assertEqual(group.id, 1)
+
+    @with_httmock(resp_create_subgroup)
+    def test_create_subgroup(self):
+        group = self.moduleUtil.createGroup({'name': "BarFoo Group", 'path': "bar-foo", "parent_id": 1})
+
+        self.assertEqual(type(group), Group)
+        self.assertEqual(group.name, "BarFoo Group")
+        self.assertEqual(group.full_path, "foo-bar/bar-foo")
+        self.assertEqual(group.id, 2)
+        self.assertEqual(group.parent_id, 1)
+
+    @with_httmock(resp_get_group)
+    def test_update_group(self):
+        group = self.gitlab_instance.groups.get(1)
+        changed, newGroup = self.moduleUtil.updateGroup(group, {'name': "BarFoo Group", "visibility": "private"})
+
+        self.assertEqual(changed, True)
+        self.assertEqual(newGroup.name, "BarFoo Group")
+        self.assertEqual(newGroup.visibility, "private")
+
+        changed, newGroup = self.moduleUtil.updateGroup(group, {'name': "BarFoo Group"})
+
+        self.assertEqual(changed, False)
+
+    @with_httmock(resp_get_group)
+    @with_httmock(resp_find_group_project)
+    @with_httmock(resp_delete_group)
+    def test_delete_group(self):
+        self.moduleUtil.existsGroup(1)
+
+        print(self.moduleUtil.groupObject.projects)
+
+        rvalue = self.moduleUtil.deleteGroup()
+
+        self.assertEqual(rvalue, None)

test/units/modules/source_control/test_gitlab_hooks.py

@@ -1,285 +1,79 @@
 # -*- coding: utf-8 -*-
-# Copyright (c) 2018 Marcus Watkins <marwatk@marcuswatkins.net>
-# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
-
-from units.compat.mock import patch
-from ansible.modules.source_control import gitlab_hooks
-from ansible.module_utils._text import to_bytes
-from ansible.module_utils import basic
-
-import pytest
-import json
-
-from units.modules.utils import set_module_args
-
-
-fake_server_state = [
-    {
-        "id": 1,
-        "url": "https://notification-server.example.com/gitlab-hook",
-        "project_id": 10,
-        "push_events": True,
-        "issues_events": True,
-        "merge_requests_events": True,
-        "tag_push_events": True,
-        "note_events": True,
-        "job_events": True,
-        "pipeline_events": True,
-        "wiki_page_events": True,
-        "enable_ssl_verification": True,
-        "created_at": "2012-10-12T17:04:47Z"
-    },
-]
-
-
-class FakeReader:
-    def __init__(self, object):
-        self.content = json.dumps(object, sort_keys=True)
-
-    def read(self):
-        return self.content
-
-
-class AnsibleExitJson(Exception):
-    """Exception class to be raised by module.exit_json and caught by the test case"""
-    pass
-
-
-class AnsibleFailJson(Exception):
-    """Exception class to be raised by module.fail_json and caught by the test case"""
-    pass
-
-
-def exit_json(*args, **kwargs):
-    """function to patch over exit_json; package return data into an exception"""
-    if 'changed' not in kwargs:
-        kwargs['changed'] = False
-    raise AnsibleExitJson(kwargs)
 
+# Copyright: (c) 2019, Guillaume Martinez (lunik@tiwabbit.fr)
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
 
-def fail_json(*args, **kwargs):
+from __future__ import absolute_import
-    """function to patch over fail_json; package return data into an exception"""
-    kwargs['failed'] = True
-    raise AnsibleFailJson(kwargs)
-
-
-@pytest.fixture
-def fetch_url_mock(mocker):
-    return mocker.patch('ansible.module_utils.gitlab.fetch_url')
-
-
-@pytest.fixture
-def module_mock(mocker):
-    return mocker.patch.multiple(basic.AnsibleModule,
-                                 exit_json=exit_json,
-                                 fail_json=fail_json)
-
-
-def test_access_token_output(capfd, fetch_url_mock, module_mock):
-    fetch_url_mock.return_value = [FakeReader(fake_server_state), {'status': 200}]
-    set_module_args({
-        'api_url': 'https://gitlab.example.com/api',
-        'access_token': 'test-access-token',
-        'project': '10',
-        'hook_url': 'https://my-ci-server.example.com/gitlab-hook',
-        'state': 'absent'
-    })
-    with pytest.raises(AnsibleExitJson) as result:
-        gitlab_hooks.main()
-
-    first_call = fetch_url_mock.call_args_list[0][1]
-    assert first_call['url'] == 'https://gitlab.example.com/api/v4/projects/10/hooks'
-    assert first_call['headers']['Authorization'] == 'Bearer test-access-token'
-    assert 'Private-Token' not in first_call['headers']
-    assert first_call['method'] == 'GET'
-
-
-def test_private_token_output(capfd, fetch_url_mock, module_mock):
-    fetch_url_mock.return_value = [FakeReader(fake_server_state), {'status': 200}]
-    set_module_args({
-        'api_url': 'https://gitlab.example.com/api',
-        'private_token': 'test-private-token',
-        'project': 'foo/bar',
-        'hook_url': 'https://my-ci-server.example.com/gitlab-hook',
-        'state': 'absent'
-    })
-    with pytest.raises(AnsibleExitJson) as result:
-        gitlab_hooks.main()
-
-    first_call = fetch_url_mock.call_args_list[0][1]
-    assert first_call['url'] == 'https://gitlab.example.com/api/v4/projects/foo%2Fbar/hooks'
-    assert first_call['headers']['Private-Token'] == 'test-private-token'
-    assert 'Authorization' not in first_call['headers']
-    assert first_call['method'] == 'GET'
-
-
-def test_bad_http_first_response(capfd, fetch_url_mock, module_mock):
-    fetch_url_mock.side_effect = [[FakeReader("Permission denied"), {'status': 403}], [FakeReader("Permission denied"), {'status': 403}]]
-    set_module_args({
-        'api_url': 'https://gitlab.example.com/api',
-        'access_token': 'test-access-token',
-        'project': '10',
-        'hook_url': 'https://my-ci-server.example.com/gitlab-hook',
-        'state': 'absent'
-    })
-    with pytest.raises(AnsibleFailJson):
-        gitlab_hooks.main()
-
 
-def test_bad_http_second_response(capfd, fetch_url_mock, module_mock):
+from ansible.modules.source_control.gitlab_hooks import GitLabHook
-    fetch_url_mock.side_effect = [[FakeReader(fake_server_state), {'status': 200}], [FakeReader("Permission denied"), {'status': 403}]]
-    set_module_args({
-        'api_url': 'https://gitlab.example.com/api',
-        'access_token': 'test-access-token',
-        'project': '10',
-        'hook_url': 'https://my-ci-server.example.com/gitlab-hook',
-        'state': 'present'
-    })
-    with pytest.raises(AnsibleFailJson):
-        gitlab_hooks.main()
 
+from .gitlab import (GitlabModuleTestCase,
+                     python_version_match_requirement,
+                     resp_get_project, resp_find_project_hook,
+                     resp_create_project_hook, resp_delete_project_hook)
 
-def test_delete_non_existing(capfd, fetch_url_mock, module_mock):
+# Gitlab module requirements
-    fetch_url_mock.return_value = [FakeReader(fake_server_state), {'status': 200}]
+if python_version_match_requirement():
-    set_module_args({
+    from gitlab.v4.objects import ProjectHook
-        'api_url': 'https://gitlab.example.com/api',
-        'access_token': 'test-access-token',
-        'project': '10',
-        'hook_url': 'https://my-ci-server.example.com/gitlab-hook',
-        'state': 'absent'
-    })
-    with pytest.raises(AnsibleExitJson) as result:
-        gitlab_hooks.main()
 
-    assert result.value.args[0]['changed'] is False
+# Unit tests requirements
+from httmock import with_httmock  # noqa
 
 
-def test_delete_existing(capfd, fetch_url_mock, module_mock):
+class TestGitlabHook(GitlabModuleTestCase):
-    fetch_url_mock.return_value = [FakeReader(fake_server_state), {'status': 200}]
+    def setUp(self):
-    set_module_args({
+        super(TestGitlabHook, self).setUp()
-        'api_url': 'https://gitlab.example.com/api',
-        'access_token': 'test-access-token',
-        'project': '10',
-        'hook_url': 'https://notification-server.example.com/gitlab-hook',
-        'state': 'absent'
-    })
-    with pytest.raises(AnsibleExitJson) as result:
-        gitlab_hooks.main()
 
-    second_call = fetch_url_mock.call_args_list[1][1]
+        self.moduleUtil = GitLabHook(module=self.mock_module, gitlab_instance=self.gitlab_instance)
 
-    assert second_call['url'] == 'https://gitlab.example.com/api/v4/projects/10/hooks/1'
+    @with_httmock(resp_get_project)
-    assert second_call['method'] == 'DELETE'
+    @with_httmock(resp_find_project_hook)
+    def test_hook_exist(self):
+        project = self.gitlab_instance.projects.get(1)
 
-    assert result.value.args[0]['changed'] is True
+        rvalue = self.moduleUtil.existsHooks(project, "http://example.com/hook")
 
+        self.assertEqual(rvalue, True)
 
-def test_add_new(capfd, fetch_url_mock, module_mock):
+        rvalue = self.moduleUtil.existsHooks(project, "http://gitlab.com/hook")
-    fetch_url_mock.return_value = [FakeReader(fake_server_state), {'status': 200}]
-    set_module_args({
-        'api_url': 'https://gitlab.example.com/api',
-        'access_token': 'test-access-token',
-        'project': '10',
-        'hook_url': 'https://my-ci-server.example.com/gitlab-hook',
-        'state': 'present'
-    })
-    with pytest.raises(AnsibleExitJson) as result:
-        gitlab_hooks.main()
 
-    second_call = fetch_url_mock.call_args_list[1][1]
+        self.assertEqual(rvalue, False)
 
-    assert second_call['url'] == 'https://gitlab.example.com/api/v4/projects/10/hooks'
+    @with_httmock(resp_get_project)
-    assert second_call['method'] == 'POST'
+    @with_httmock(resp_create_project_hook)
-    assert second_call['data'] == ('{"enable_ssl_verification": false, "issues_events": false, "job_events": false, '
+    def test_create_hook(self):
-                                   '"merge_requests_events": false, "note_events": false, "pipeline_events": false, "push_events": true, "tag_push_events": '
+        project = self.gitlab_instance.projects.get(1)
-                                   'false, "token": null, "url": "https://my-ci-server.example.com/gitlab-hook", "wiki_page_events": false}')
-    assert result.value.args[0]['changed'] is True
 
+        hook = self.moduleUtil.createHook(project, {"url": "http://example.com/hook"})
 
-def test_update_existing(capfd, fetch_url_mock, module_mock):
+        self.assertEqual(type(hook), ProjectHook)
-    fetch_url_mock.return_value = [FakeReader(fake_server_state), {'status': 200}]
+        self.assertEqual(hook.url, "http://example.com/hook")
-    set_module_args({
-        'api_url': 'https://gitlab.example.com/api',
-        'access_token': 'test-access-token',
-        'project': '10',
-        'hook_url': 'https://notification-server.example.com/gitlab-hook',
-        'push_events': 'yes',
-        'issues_events': 'yes',
-        'merge_requests_events': 'yes',
-        'tag_push_events': 'yes',
-        'note_events': 'yes',
-        'job_events': 'yes',
-        'pipeline_events': 'yes',
-        'wiki_page_events': 'no',
-        'enable_ssl_verification': 'yes',
-        'state': 'present'
-    })
-    with pytest.raises(AnsibleExitJson) as result:
-        gitlab_hooks.main()
 
-    second_call = fetch_url_mock.call_args_list[1][1]
+    @with_httmock(resp_get_project)
+    @with_httmock(resp_find_project_hook)
+    def test_update_hook(self):
+        project = self.gitlab_instance.projects.get(1)
+        hook = self.moduleUtil.findHook(project, "http://example.com/hook")
 
-    assert second_call['url'] == 'https://gitlab.example.com/api/v4/projects/10/hooks/1'
+        changed, newHook = self.moduleUtil.updateHook(hook, {"url": "http://gitlab.com/hook"})
-    assert second_call['method'] == 'PUT'
-    assert second_call['data'] == ('{"enable_ssl_verification": true, "issues_events": true, "job_events": true, '
-                                   '"merge_requests_events": true, "note_events": true, "pipeline_events": true, "push_events": true, "tag_push_events": '
-                                   'true, "token": null, "url": "https://notification-server.example.com/gitlab-hook", "wiki_page_events": false}')
-    assert result.value.args[0]['changed'] is True
 
+        self.assertEqual(changed, True)
+        self.assertEqual(type(newHook), ProjectHook)
+        self.assertEqual(newHook.url, "http://gitlab.com/hook")
 
-def test_unchanged_existing(capfd, fetch_url_mock, module_mock):
+        changed, newHook = self.moduleUtil.updateHook(hook, {"url": "http://gitlab.com/hook"})
-    fetch_url_mock.return_value = [FakeReader(fake_server_state), {'status': 200}]
-    set_module_args({
-        'api_url': 'https://gitlab.example.com/api',
-        'access_token': 'test-access-token',
-        'project': '10',
-        'hook_url': 'https://notification-server.example.com/gitlab-hook',
-        'push_events': 'yes',
-        'issues_events': 'yes',
-        'merge_requests_events': 'yes',
-        'tag_push_events': 'yes',
-        'note_events': 'yes',
-        'job_events': 'yes',
-        'pipeline_events': 'yes',
-        'wiki_page_events': 'yes',
-        'enable_ssl_verification': 'yes',
-        'state': 'present'
-    })
-    with pytest.raises(AnsibleExitJson) as result:
-        gitlab_hooks.main()
 
-    assert result.value.args[0]['changed'] is False
+        self.assertEqual(changed, False)
-    assert fetch_url_mock.call_count == 1
+        self.assertEqual(newHook.url, "http://gitlab.com/hook")
 
+    @with_httmock(resp_get_project)
+    @with_httmock(resp_find_project_hook)
+    @with_httmock(resp_delete_project_hook)
+    def test_delete_hook(self):
+        project = self.gitlab_instance.projects.get(1)
 
-def test_unchanged_existing_with_token(capfd, fetch_url_mock, module_mock):
+        self.moduleUtil.existsHooks(project, "http://example.com/hook")
-    fetch_url_mock.return_value = [FakeReader(fake_server_state), {'status': 200}]
-    set_module_args({
-        'api_url': 'https://gitlab.example.com/api',
-        'access_token': 'test-access-token',
-        'project': '10',
-        'hook_url': 'https://notification-server.example.com/gitlab-hook',
-        'push_events': 'yes',
-        'issues_events': 'yes',
-        'merge_requests_events': 'yes',
-        'tag_push_events': 'yes',
-        'note_events': 'yes',
-        'job_events': 'yes',
-        'pipeline_events': 'yes',
-        'wiki_page_events': 'yes',
-        'enable_ssl_verification': 'yes',
-        'state': 'present',
-        'token': 'secret-token',
-    })
-    with pytest.raises(AnsibleExitJson) as result:
-        gitlab_hooks.main()
 
-    second_call = fetch_url_mock.call_args_list[1][1]
+        rvalue = self.moduleUtil.deleteHook()
 
-    assert second_call['url'] == 'https://gitlab.example.com/api/v4/projects/10/hooks/1'
+        self.assertEqual(rvalue, None)
-    assert second_call['method'] == 'PUT'
-    assert second_call['data'] == ('{"enable_ssl_verification": true, "issues_events": true, "job_events": true, '
-                                   '"merge_requests_events": true, "note_events": true, "pipeline_events": true, "push_events": true, '
-                                   '"tag_push_events": true, "token": "secret-token", "url": "https://notification-server.example.com/gitlab-hook", '
-                                   '"wiki_page_events": true}')
-    assert result.value.args[0]['changed'] is True

test/units/modules/source_control/test_gitlab_project.py

@@ -1,82 +1,78 @@
 # -*- coding: utf-8 -*-
-# Copyright (c) 2018 Pierre-Louis Bonicoli <pierre-louis@libregerbil.fr>
+
+# Copyright: (c) 2019, Guillaume Martinez (lunik@tiwabbit.fr)
 # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
 
-import json
+from __future__ import absolute_import
-
+
-from units.compat.mock import MagicMock
+from ansible.modules.source_control.gitlab_project import GitLabProject
-from ansible.modules.source_control import gitlab_project
+
-
+from .gitlab import (GitlabModuleTestCase,
-import pytest
+                     python_version_match_requirement,
-
+                     resp_get_group, resp_get_project_by_name, resp_create_project,
-
+                     resp_get_project, resp_delete_project, resp_get_user)
-@pytest.fixture
+
-def patch_gitlab_project(mocker):
+# Gitlab module requirements
-    mocker.patch.object(gitlab_project, 'HAS_GITLAB_PACKAGE', mocker.PropertyMock(return_value=True))
+if python_version_match_requirement():
-
+    from gitlab.v4.objects import Project
-
+
-@pytest.mark.parametrize('patch_ansible_module', [{}], indirect=['patch_ansible_module'])
+# Unit tests requirements
-@pytest.mark.usefixtures('patch_ansible_module')
+from httmock import with_httmock  # noqa
-def test_without_required_parameters(capfd):
+
-    """Failure must occurs when all parameters are missing"""
+
-
+class TestGitlabProject(GitlabModuleTestCase):
-    with pytest.raises(SystemExit):
+    @with_httmock(resp_get_user)
-        gitlab_project.main()
+    def setUp(self):
-    out, err = capfd.readouterr()
+        super(TestGitlabProject, self).setUp()
-    results = json.loads(out)
+
-    assert results['failed']
+        self.gitlab_instance.user = self.gitlab_instance.users.get(1)
-    assert 'missing required arguments' in results['msg']
+        self.moduleUtil = GitLabProject(module=self.mock_module, gitlab_instance=self.gitlab_instance)
-
+
-
+    @with_httmock(resp_get_group)
-TEST_CASES = [
+    @with_httmock(resp_get_project_by_name)
-    [
+    def test_project_exist(self):
-        {
+        group = self.gitlab_instance.groups.get(1)
-            'server_url': 'http://gitlab.test/gitlab',
+
-            'validate_certs': True,
+        rvalue = self.moduleUtil.existsProject(group, "diaspora-client")
-            'login_user': 'john',
+
-            'login_token': 'TOKEN',
+        self.assertEqual(rvalue, True)
-            'name': 'new_test_repo',
+
-            'group': 'my_repo_group',
+        rvalue = self.moduleUtil.existsProject(group, "missing-project")
-            'public': True,
+
-            'visibility_level': 20,
+        self.assertEqual(rvalue, False)
-            'issues_enabled': False,
+
-            'wiki_enabled': True,
+    @with_httmock(resp_get_group)
-            'snippets_enabled': True,
+    @with_httmock(resp_create_project)
-            'import_url': 'http://gitlab.test/gitlab/gitrepothatdoesnotexist.git',
+    def test_create_project(self):
-            'state': 'present'
+        group = self.gitlab_instance.groups.get(1)
-        },
+        project = self.moduleUtil.createProject(group, {"name": "Diaspora Client", "path": "diaspora-client", "namespace_id": group.id})
-        {
+
-            'msg': "Failed to create project 'new_test_repo'",
+        self.assertEqual(type(project), Project)
-            'failed': True,
+        self.assertEqual(project.name, "Diaspora Client")
-        }
+
-    ],
+    @with_httmock(resp_get_project)
-]
+    def test_update_project(self):
-
+        project = self.gitlab_instance.projects.get(1)
-
+
-@pytest.mark.parametrize('patch_ansible_module, testcase', TEST_CASES, indirect=['patch_ansible_module'])
+        changed, newProject = self.moduleUtil.updateProject(project, {"name": "New Name"})
-@pytest.mark.usefixtures('patch_ansible_module')
+
-def test_fail_if_url_import_doesnt_exist(mocker, capfd, patch_gitlab_project, testcase):
+        self.assertEqual(changed, True)
-    """ Test for #36495
+        self.assertEqual(type(newProject), Project)
-
+        self.assertEqual(newProject.name, "New Name")
-    Ensure errors are reported (meaning task report a failure),
+
-    for example when url_import doesn't exist, an error must occur.
+        changed, newProject = self.moduleUtil.updateProject(project, {"name": "New Name"})
-    """
+
-
+        self.assertEqual(changed, False)
-    git = MagicMock()
+        self.assertEqual(newProject.name, "New Name")
-    git.createprojectuser.return_value = False
+
-
+    @with_httmock(resp_get_group)
-    gitlab = MagicMock()
+    @with_httmock(resp_get_project_by_name)
-    gitlab.Gitlab.return_value = git
+    @with_httmock(resp_delete_project)
-    gitlab_project.gitlab = gitlab
+    def test_delete_project(self):
-
+        group = self.gitlab_instance.groups.get(1)
-    with pytest.raises(SystemExit):
+
-        gitlab_project.main()
+        self.moduleUtil.existsProject(group, "diaspora-client")
-
+
-    # Check that 1. createprojectuser method has been called 2. with expected parameter
+        rvalue = self.moduleUtil.deleteProject()
-    assert git.createprojectuser.call_count == 1
+
-    assert git.createprojectuser.call_args[1]['import_url'] == 'http://gitlab.test/gitlab/gitrepothatdoesnotexist.git'
+        self.assertEqual(rvalue, None)
-
-    out, err = capfd.readouterr()
-    results = json.loads(out)
-    assert results.get('failed') == testcase.get('failed')
-    assert results['msg'] == testcase['msg']

test/units/modules/source_control/test_gitlab_runner.py

@@ -0,0 +1,71 @@
+# -*- coding: utf-8 -*-
+
+# Copyright: (c) 2019, Guillaume Martinez (lunik@tiwabbit.fr)
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import absolute_import
+
+from ansible.modules.source_control.gitlab_runner import GitLabRunner
+
+from .gitlab import (GitlabModuleTestCase,
+                     python_version_match_requirement,
+                     resp_find_runners, resp_get_runner,
+                     resp_create_runner, resp_delete_runner)
+
+# Gitlab module requirements
+if python_version_match_requirement():
+    from gitlab.v4.objects import Runner
+
+# Unit tests requirements
+from httmock import with_httmock  # noqa
+
+
+class TestGitlabRunner(GitlabModuleTestCase):
+    def setUp(self):
+        super(TestGitlabRunner, self).setUp()
+
+        self.moduleUtil = GitLabRunner(module=self.mock_module, gitlab_instance=self.gitlab_instance)
+
+    @with_httmock(resp_find_runners)
+    @with_httmock(resp_get_runner)
+    def test_runner_exist(self):
+        rvalue = self.moduleUtil.existsRunner("test-1-20150125")
+
+        self.assertEqual(rvalue, True)
+
+        rvalue = self.moduleUtil.existsRunner("test-3-00000000")
+
+        self.assertEqual(rvalue, False)
+
+    @with_httmock(resp_create_runner)
+    def test_create_runner(self):
+        runner = self.moduleUtil.createRunner({"token": "token", "description": "test-1-20150125"})
+
+        self.assertEqual(type(runner), Runner)
+        self.assertEqual(runner.description, "test-1-20150125")
+
+    @with_httmock(resp_find_runners)
+    @with_httmock(resp_get_runner)
+    def test_update_runner(self):
+        runner = self.moduleUtil.findRunner("test-1-20150125")
+
+        changed, newRunner = self.moduleUtil.updateRunner(runner, {"description": "Runner description"})
+
+        self.assertEqual(changed, True)
+        self.assertEqual(type(newRunner), Runner)
+        self.assertEqual(newRunner.description, "Runner description")
+
+        changed, newRunner = self.moduleUtil.updateRunner(runner, {"description": "Runner description"})
+
+        self.assertEqual(changed, False)
+        self.assertEqual(newRunner.description, "Runner description")
+
+    @with_httmock(resp_find_runners)
+    @with_httmock(resp_get_runner)
+    @with_httmock(resp_delete_runner)
+    def test_delete_runner(self):
+        self.moduleUtil.existsRunner("test-1-20150125")
+
+        rvalue = self.moduleUtil.deleteRunner()
+
+        self.assertEqual(rvalue, None)

test/units/modules/source_control/test_gitlab_user.py

@@ -0,0 +1,133 @@
+# -*- coding: utf-8 -*-
+
+# Copyright: (c) 2019, Guillaume Martinez (lunik@tiwabbit.fr)
+# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import absolute_import
+
+from ansible.modules.source_control.gitlab_user import GitLabUser
+
+from .gitlab import (GitlabModuleTestCase,
+                     python_version_match_requirement,
+                     resp_find_user, resp_get_user, resp_get_user_keys,
+                     resp_create_user_keys, resp_create_user, resp_delete_user,
+                     resp_get_member, resp_get_group, resp_add_member,
+                     resp_update_member, resp_get_member)
+
+# Gitlab module requirements
+if python_version_match_requirement():
+    from gitlab.v4.objects import User
+
+# Unit tests requirements
+from httmock import with_httmock  # noqa
+
+
+class TestGitlabUser(GitlabModuleTestCase):
+    def setUp(self):
+        super(TestGitlabUser, self).setUp()
+
+        self.moduleUtil = GitLabUser(module=self.mock_module, gitlab_instance=self.gitlab_instance)
+
+    @with_httmock(resp_find_user)
+    def test_exist_user(self):
+        rvalue = self.moduleUtil.existsUser("john_smith")
+
+        self.assertEqual(rvalue, True)
+
+        rvalue = self.moduleUtil.existsUser("paul_smith")
+
+        self.assertEqual(rvalue, False)
+
+    @with_httmock(resp_find_user)
+    def test_find_user(self):
+        user = self.moduleUtil.findUser("john_smith")
+
+        self.assertEqual(type(user), User)
+        self.assertEqual(user.name, "John Smith")
+        self.assertEqual(user.id, 1)
+
+    @with_httmock(resp_create_user)
+    def test_create_user(self):
+        user = self.moduleUtil.createUser({'email': 'john@example.com', 'password': 's3cur3s3cr3T',
+                                           'username': 'john_smith', 'name': 'John Smith'})
+        self.assertEqual(type(user), User)
+        self.assertEqual(user.name, "John Smith")
+        self.assertEqual(user.id, 1)
+
+    @with_httmock(resp_get_user)
+    def test_update_user(self):
+        user = self.gitlab_instance.users.get(1)
+        changed, newUser = self.moduleUtil.updateUser(user, {'name': "Jack Smith", "is_admin": "true"})
+
+        self.assertEqual(changed, True)
+        self.assertEqual(newUser.name, "Jack Smith")
+        self.assertEqual(newUser.is_admin, "true")
+
+        changed, newUser = self.moduleUtil.updateUser(user, {'name': "Jack Smith"})
+
+        self.assertEqual(changed, False)
+
+    @with_httmock(resp_find_user)
+    @with_httmock(resp_delete_user)
+    def test_delete_user(self):
+        self.moduleUtil.existsUser("john_smith")
+        rvalue = self.moduleUtil.deleteUser()
+
+        self.assertEqual(rvalue, None)
+
+    @with_httmock(resp_get_user)
+    @with_httmock(resp_get_user_keys)
+    def test_sshkey_exist(self):
+        user = self.gitlab_instance.users.get(1)
+
+        exist = self.moduleUtil.sshKeyExists(user, "Public key")
+        self.assertEqual(exist, True)
+
+        notExist = self.moduleUtil.sshKeyExists(user, "Private key")
+        self.assertEqual(notExist, False)
+
+    @with_httmock(resp_get_user)
+    @with_httmock(resp_create_user_keys)
+    @with_httmock(resp_get_user_keys)
+    def test_create_sshkey(self):
+        user = self.gitlab_instance.users.get(1)
+
+        rvalue = self.moduleUtil.addSshKeyToUser(user, {
+            'name': "Public key",
+            'file': "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAiPWx6WM4lhHNedGfBpPJNPpZ7yKu+dnn1SJe"
+                    "jgt4596k6YjzGGphH2TUxwKzxcKDKKezwkpfnxPkSMkuEspGRt/aZZ9wa++Oi7Qkr8prgHc4"
+                    "soW6NUlfDzpvZK2H5E7eQaSeP3SAwGmQKUFHCddNaP0L+hM7zhFNzjFvpaMgJw0="})
+        self.assertEqual(rvalue, False)
+
+        rvalue = self.moduleUtil.addSshKeyToUser(user, {
+            'name': "Private key",
+            'file': "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDA1YotVDm2mAyk2tPt4E7AHm01sS6JZmcU"
+                    "dRuSuA5zszUJzYPPUSRAX3BCgTqLqYx//UuVncK7YqLVSbbwjKR2Ez5lISgCnVfLVEXzwhv+"
+                    "xawxKWmI7hJ5S0tOv6MJ+IxyTa4xcKwJTwB86z22n9fVOQeJTR2dSOH1WJrf0PvRk+KVNY2j"
+                    "TiGHTi9AIjLnyD/jWRpOgtdfkLRc8EzAWrWlgNmH2WOKBw6za0az6XoG75obUdFVdW3qcD0x"
+                    "c809OHLi7FDf+E7U4wiZJCFuUizMeXyuK/SkaE1aee4Qp5R4dxTR4TP9M1XAYkf+kF0W9srZ+mhF069XD/zhUPJsvwEF"})
+        self.assertEqual(rvalue, True)
+
+    @with_httmock(resp_get_group)
+    @with_httmock(resp_get_member)
+    def test_find_member(self):
+        group = self.gitlab_instance.groups.get(1)
+
+        user = self.moduleUtil.findMember(group, 1)
+        self.assertEqual(user.username, "raymond_smith")
+
+    @with_httmock(resp_get_user)
+    @with_httmock(resp_get_group)
+    @with_httmock(resp_get_group)
+    @with_httmock(resp_get_member)
+    @with_httmock(resp_add_member)
+    @with_httmock(resp_update_member)
+    def test_assign_user_to_group(self):
+        group = self.gitlab_instance.groups.get(1)
+        user = self.gitlab_instance.users.get(1)
+
+        rvalue = self.moduleUtil.assignUserToGroup(user, group.id, "developer")
+        self.assertEqual(rvalue, False)
+
+        rvalue = self.moduleUtil.assignUserToGroup(user, group.id, "guest")
+        self.assertEqual(rvalue, True)