Enhancement to gitlab_group_members to accept user lists as input (#3047)

* - fix to issue 3041
- add func to work with user lists
- add func to set members to the ones give

* Added version_added to new parameter

* fixed elements in definition of gitlab_users nad wrong import in gitlab_users

* linter issues fixed

* added list elelements to argument_spec

* More whitspeaces for the linter

* Update plugins/modules/source_control/gitlab/gitlab_group_members.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* adapted changelog

* removed bugfix (other PR), changes due to review

* changed input handling according to review

* Fixed test findings

* Added list of dict to allow for specifying user/access_level tuples

* corrected doc section

* fixed parameter definitions

* removed strange additional import

* Update changelogs/fragments/3041-gitlab_x_members_fix_and_enhancement.yml

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update plugins/modules/source_control/gitlab/gitlab_group_members.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update plugins/modules/source_control/gitlab/gitlab_project_members.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update plugins/modules/source_control/gitlab/gitlab_group_members.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update plugins/modules/source_control/gitlab/gitlab_group_members.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* fixed required if

* Update plugins/modules/source_control/gitlab/gitlab_group_members.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Added suggestions from PR

* fixed identation problem

* Update plugins/modules/source_control/gitlab/gitlab_group_members.py

Co-authored-by: Zainab Alsaffar <za5775@rit.edu>

* Update plugins/modules/source_control/gitlab/gitlab_group_members.py

Co-authored-by: Zainab Alsaffar <za5775@rit.edu>

* Update plugins/modules/source_control/gitlab/gitlab_group_members.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update plugins/modules/source_control/gitlab/gitlab_group_members.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Recommended changes from discussionst

* Fixed issues from automatic tests

* added missing metaclass due to test finding

* added integration tests

* Update plugins/modules/source_control/gitlab/gitlab_group_members.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update plugins/modules/source_control/gitlab/gitlab_group_members.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update plugins/modules/source_control/gitlab/gitlab_group_members.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update plugins/modules/source_control/gitlab/gitlab_group_members.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update plugins/modules/source_control/gitlab/gitlab_group_members.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update plugins/modules/source_control/gitlab/gitlab_group_members.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update plugins/modules/source_control/gitlab/gitlab_group_members.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* fixed optimization for only one user

* Reverted gitlab_project_members to original version - changes will be done in a separate branch

* added examples for new functionality

* - fixed changelog after reverting gitlab_project_memebers
- fully reverted gitlab_project_members

* Fixed error handling: when single users are not updateable in bulk mode the exception should not stop the code flow but document the problem in the result.

* Better error handling

* on error give username, not gitlab numeric userid

* Fixed broken check_mode

* Update plugins/modules/source_control/gitlab/gitlab_group_members.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Change from review

Co-authored-by: Max Bidlingmaier <Max-Florian.Bidlingmaier@sap.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Zainab Alsaffar <za5775@rit.edu>

changelogs/fragments/3041-gitlab_x_members_fix_and_enhancement.yml

@@ -0,0 +1,3 @@
+minor_changes:
+- gitlab_group_members - ``gitlab_user`` can now also be a list of users (https://github.com/ansible-collections/community.general/pull/3047).
+- gitlab_group_members - added functionality to set all members exactly as given (https://github.com/ansible-collections/community.general/pull/3047).

plugins/modules/source_control/gitlab/gitlab_group_members.py

@@ -32,15 +32,38 @@ options:
         type: str
     gitlab_user:
         description:
-            - The username of the member to add to/remove from the GitLab group.
+            - A username or a list of usernames to add to/remove from the GitLab group.
-        required: true
+            - Mutually exclusive with I(gitlab_users_access).
+        type: list
+        elements: str
+    access_level:
+        description:
+            - The access level for the user.
+            - Required if I(state=present), user state is set to present.
+            - Mutually exclusive with I(gitlab_users_access).
         type: str
+        choices: ['guest', 'reporter', 'developer', 'maintainer', 'owner']
+    gitlab_users_access:
+        description:
+            - Provide a list of user to access level mappings.
+            - Every dictionary in this list specifies a user (by username) and the access level the user should have.
+            - Mutually exclusive with I(gitlab_user) and I(access_level).
+            - Use together with I(purge_users) to remove all users not specified here from the group.
+        type: list
+        elements: dict
+        suboptions:
+            name:
+                description: A username or a list of usernames to add to/remove from the GitLab group.
+                type: str
+                required: true
             access_level:
                 description:
                     - The access level for the user.
                     - Required if I(state=present), user state is set to present.
                 type: str
                 choices: ['guest', 'reporter', 'developer', 'maintainer', 'owner']
+                required: true
+        version_added: 3.6.0
     state:
         description:
             - State of the member in the group.
@@ -49,6 +72,15 @@ options:
         choices: ['present', 'absent']
         default: 'present'
         type: str
+    purge_users:
+        description:
+            - Adds/remove users of the given access_level to match the given gitlab_user/gitlab_users_access list.
+              If omitted do not purge orphaned members.
+            - Is only used when I(state=present).
+        type: list
+        elements: str
+        choices: ['guest', 'reporter', 'developer', 'maintainer', 'owner']
+        version_added: 3.6.0
 notes:
     - Supports C(check_mode).
 '''
@@ -70,6 +102,51 @@ EXAMPLES = r'''
     gitlab_group: groupname
     gitlab_user: username
     state: absent
+
+- name: Add a list of Users to A GitLab Group
+  gitlab_group_members:
+    api_url: 'https://gitlab.example.com'
+    api_token: 'Your-Private-Token'
+    gitlab_group: groupname
+    gitlab_user:
+      - user1
+      - user2
+    access_level: developer
+    state: present
+
+- name: Add a list of Users with Dedicated Access Levels to A GitLab Group
+  gitlab_group_members:
+    api_url: 'https://gitlab.example.com'
+    api_token: 'Your-Private-Token'
+    gitlab_group: groupname
+    gitlab_users_access:
+      - name: user1
+        access_level: developer
+      - name: user2
+        access_level: maintainer
+    state: present
+
+- name: Add a user, remove all others which might be on this access level
+  gitlab_group_members:
+    api_url: 'https://gitlab.example.com'
+    api_token: 'Your-Private-Token'
+    gitlab_group: groupname
+    gitlab_user: username
+    access_level: developer
+    pruge_users: developer
+    state: present
+
+- name: Remove a list of Users with Dedicated Access Levels to A GitLab Group
+  gitlab_group_members:
+    api_url: 'https://gitlab.example.com'
+    api_token: 'Your-Private-Token'
+    gitlab_group: groupname
+    gitlab_users_access:
+      - name: user1
+        access_level: developer
+      - name: user2
+        access_level: maintainer
+    state: absent
 '''
 
 RETURN = r''' # '''
@@ -111,6 +188,17 @@ class GitLabGroup(object):
         group = self._gitlab.groups.get(gitlab_group_id)
         return group.members.list(all=True)
 
+    # get single member in a group by user name
+    def get_member_in_a_group(self, gitlab_group_id, gitlab_user_id):
+        member = None
+        group = self._gitlab.groups.get(gitlab_group_id)
+        try:
+            member = group.members.get(gitlab_user_id)
+            if member:
+                return member
+        except gitlab.exceptions.GitlabGetError as e:
+            return None
+
     # check if the user is a member of the group
     def is_user_a_member(self, members, gitlab_user_id):
         for member in members:
@@ -120,28 +208,15 @@ class GitLabGroup(object):
 
     # add user to a group
     def add_member_to_group(self, gitlab_user_id, gitlab_group_id, access_level):
-        try:
         group = self._gitlab.groups.get(gitlab_group_id)
         add_member = group.members.create(
             {'user_id': gitlab_user_id, 'access_level': access_level})
 
-            if add_member:
-                return add_member.username
-
-        except (gitlab.exceptions.GitlabCreateError) as e:
-            self._module.fail_json(
-                msg="Failed to add member to the Group, Group ID %s: %s" % (gitlab_group_id, e))
-
     # remove user from a group
     def remove_user_from_group(self, gitlab_user_id, gitlab_group_id):
-        try:
         group = self._gitlab.groups.get(gitlab_group_id)
         group.members.delete(gitlab_user_id)
 
-        except (gitlab.exceptions.GitlabDeleteError) as e:
-            self._module.fail_json(
-                msg="Failed to remove member from GitLab group, ID %s: %s" % (gitlab_group_id, e))
-
     # get user's access level
     def get_user_access_level(self, members, gitlab_user_id):
         for member in members:
@@ -152,12 +227,8 @@ class GitLabGroup(object):
     def update_user_access_level(self, members, gitlab_user_id, access_level):
         for member in members:
             if member.id == gitlab_user_id:
-                try:
                 member.access_level = access_level
                 member.save()
-                except (gitlab.exceptions.GitlabCreateError) as e:
-                    self._module.fail_json(
-                        msg="Failed to update the access level for the member, %s: %s" % (gitlab_user_id, e))
 
 
 def main():
@@ -165,9 +236,18 @@ def main():
     argument_spec.update(dict(
         api_token=dict(type='str', required=True, no_log=True),
         gitlab_group=dict(type='str', required=True),
-        gitlab_user=dict(type='str', required=True),
+        gitlab_user=dict(type='list', elements='str'),
         state=dict(type='str', default='present', choices=['present', 'absent']),
-        access_level=dict(type='str', required=False, choices=['guest', 'reporter', 'developer', 'maintainer', 'owner'])
+        access_level=dict(type='str', choices=['guest', 'reporter', 'developer', 'maintainer', 'owner']),
+        purge_users=dict(type='list', elements='str', choices=['guest', 'reporter', 'developer', 'maintainer', 'owner']),
+        gitlab_users_access=dict(
+            type='list',
+            elements='dict',
+            options=dict(
+                name=dict(type='str', required=True),
+                access_level=dict(type='str', choices=['guest', 'reporter', 'developer', 'maintainer', 'owner'], required=True),
+            )
+        ),
     ))
 
     module = AnsibleModule(
@@ -175,15 +255,19 @@ def main():
         mutually_exclusive=[
             ['api_username', 'api_token'],
             ['api_password', 'api_token'],
+            ['gitlab_user', 'gitlab_users_access'],
+            ['access_level', 'gitlab_users_access'],
         ],
         required_together=[
             ['api_username', 'api_password'],
+            ['gitlab_user', 'access_level'],
         ],
         required_one_of=[
             ['api_username', 'api_token'],
+            ['gitlab_user', 'gitlab_users_access'],
         ],
         required_if=[
-            ['state', 'present', ['access_level']],
+            ['state', 'present', ['access_level', 'gitlab_users_access'], True],
         ],
         supports_check_mode=True,
     )
@@ -191,13 +275,6 @@ def main():
     if not HAS_PY_GITLAB:
         module.fail_json(msg=missing_required_lib('python-gitlab', url='https://python-gitlab.readthedocs.io/en/stable/'), exception=GITLAB_IMP_ERR)
 
-    gitlab_group = module.params['gitlab_group']
-    gitlab_user = module.params['gitlab_user']
-    state = module.params['state']
-    access_level = module.params['access_level']
-
-    # convert access level string input to int
-    if access_level:
     access_level_int = {
         'guest': gitlab.GUEST_ACCESS,
         'reporter': gitlab.REPORTER_ACCESS,
@@ -206,57 +283,158 @@ def main():
         'owner': gitlab.OWNER_ACCESS
     }
 
-        access_level = access_level_int[access_level]
+    gitlab_group = module.params['gitlab_group']
+    state = module.params['state']
+    access_level = module.params['access_level']
+    purge_users = module.params['purge_users']
+
+    if purge_users:
+        purge_users = [access_level_int[level] for level in purge_users]
 
     # connect to gitlab server
     gl = gitlabAuthentication(module)
 
     group = GitLabGroup(module, gl)
 
-    gitlab_user_id = group.get_user_id(gitlab_user)
     gitlab_group_id = group.get_group_id(gitlab_group)
 
     # group doesn't exist
     if not gitlab_group_id:
         module.fail_json(msg="group '%s' not found." % gitlab_group)
 
+    members = []
+    if module.params['gitlab_user'] is not None:
+        gitlab_users_access = []
+        gitlab_users = module.params['gitlab_user']
+        for gl_user in gitlab_users:
+            gitlab_users_access.append({'name': gl_user, 'access_level': access_level_int[access_level] if access_level else None})
+    elif module.params['gitlab_users_access'] is not None:
+        gitlab_users_access = module.params['gitlab_users_access']
+        for user_level in gitlab_users_access:
+            user_level['access_level'] = access_level_int[user_level['access_level']]
+
+    if len(gitlab_users_access) == 1 and not purge_users:
+        # only single user given
+        members = [group.get_member_in_a_group(gitlab_group_id, group.get_user_id(gitlab_users_access[0]['name']))]
+        if members[0] is None:
+            members = []
+    elif len(gitlab_users_access) > 1 or purge_users:
+        # list of users given
+        members = group.get_members_in_a_group(gitlab_group_id)
+    else:
+        module.exit_json(changed='OK', result="Nothing to do, please give at least one user or set purge_users true.",
+                         result_data=[])
+
+    changed = False
+    error = False
+    changed_users = []
+    changed_data = []
+
+    for gitlab_user in gitlab_users_access:
+        gitlab_user_id = group.get_user_id(gitlab_user['name'])
+
         # user doesn't exist
         if not gitlab_user_id:
             if state == 'absent':
-            module.exit_json(changed=False, result="user '%s' not found, and thus also not part of the group" % gitlab_user)
+                changed_users.append("user '%s' not found, and thus also not part of the group" % gitlab_user['name'])
+                changed_data.append({'gitlab_user': gitlab_user['name'], 'result': 'OK',
+                                     'msg': "user '%s' not found, and thus also not part of the group" % gitlab_user['name']})
             else:
-            module.fail_json(msg="user '%s' not found." % gitlab_user)
+                error = True
+                changed_users.append("user '%s' not found." % gitlab_user['name'])
+                changed_data.append({'gitlab_user': gitlab_user['name'], 'result': 'FAILED',
+                                     'msg': "user '%s' not found." % gitlab_user['name']})
+            continue
 
-    members = group.get_members_in_a_group(gitlab_group_id)
         is_user_a_member = group.is_user_a_member(members, gitlab_user_id)
 
         # check if the user is a member in the group
         if not is_user_a_member:
             if state == 'present':
                 # add user to the group
+                try:
                     if not module.check_mode:
-                group.add_member_to_group(gitlab_user_id, gitlab_group_id, access_level)
+                        group.add_member_to_group(gitlab_user_id, gitlab_group_id, gitlab_user['access_level'])
-            module.exit_json(changed=True, result="Successfully added user '%s' to the group." % gitlab_user)
+                    changed = True
+                    changed_users.append("Successfully added user '%s' to group" % gitlab_user['name'])
+                    changed_data.append({'gitlab_user': gitlab_user['name'], 'result': 'CHANGED',
+                                         'msg': "Successfully added user '%s' to group" % gitlab_user['name']})
+                except (gitlab.exceptions.GitlabCreateError) as e:
+                    error = True
+                    changed_users.append("Failed to updated the access level for the user, '%s'" % gitlab_user['name'])
+                    changed_data.append({'gitlab_user': gitlab_user['name'], 'result': 'FAILED',
+                                         'msg': "Not allowed to add the access level for the member, %s: %s" % (gitlab_user['name'], e)})
             # state as absent
             else:
-            module.exit_json(changed=False, result="User, '%s', is not a member in the group. No change to report" % gitlab_user)
+                changed_users.append("User, '%s', is not a member in the group. No change to report" % gitlab_user['name'])
+                changed_data.append({'gitlab_user': gitlab_user['name'], 'result': 'OK',
+                                     'msg': "User, '%s', is not a member in the group. No change to report" % gitlab_user['name']})
         # in case that a user is a member
         else:
             if state == 'present':
                 # compare the access level
                 user_access_level = group.get_user_access_level(members, gitlab_user_id)
-            if user_access_level == access_level:
+                if user_access_level == gitlab_user['access_level']:
-                module.exit_json(changed=False, result="User, '%s', is already a member in the group. No change to report" % gitlab_user)
+                    changed_users.append("User, '%s', is already a member in the group. No change to report" % gitlab_user['name'])
+                    changed_data.append({'gitlab_user': gitlab_user['name'], 'result': 'OK',
+                                         'msg': "User, '%s', is already a member in the group. No change to report" % gitlab_user['name']})
                 else:
                     # update the access level for the user
+                    try:
                         if not module.check_mode:
-                    group.update_user_access_level(members, gitlab_user_id, access_level)
+                            group.update_user_access_level(members, gitlab_user_id, gitlab_user['access_level'])
-                module.exit_json(changed=True, result="Successfully updated the access level for the user, '%s'" % gitlab_user)
+                        changed = True
+                        changed_users.append("Successfully updated the access level for the user, '%s'" % gitlab_user['name'])
+                        changed_data.append({'gitlab_user': gitlab_user['name'], 'result': 'CHANGED',
+                                             'msg': "Successfully updated the access level for the user, '%s'" % gitlab_user['name']})
+                    except (gitlab.exceptions.GitlabUpdateError) as e:
+                        error = True
+                        changed_users.append("Failed to updated the access level for the user, '%s'" % gitlab_user['name'])
+                        changed_data.append({'gitlab_user': gitlab_user['name'], 'result': 'FAILED',
+                                             'msg': "Not allowed to update the access level for the member, %s: %s" % (gitlab_user['name'], e)})
             else:
                 # remove the user from the group
+                try:
                     if not module.check_mode:
                         group.remove_user_from_group(gitlab_user_id, gitlab_group_id)
-            module.exit_json(changed=True, result="Successfully removed user, '%s', from the group" % gitlab_user)
+                    changed = True
+                    changed_users.append("Successfully removed user, '%s', from the group" % gitlab_user['name'])
+                    changed_data.append({'gitlab_user': gitlab_user['name'], 'result': 'CHANGED',
+                                         'msg': "Successfully removed user, '%s', from the group" % gitlab_user['name']})
+                except (gitlab.exceptions.GitlabDeleteError) as e:
+                    error = True
+                    changed_users.append("Failed to removed user, '%s', from the group" % gitlab_user['name'])
+                    changed_data.append({'gitlab_user': gitlab_user['name'], 'result': 'FAILED',
+                                         'msg': "Failed to remove user, '%s' from the group: %s" % (gitlab_user['name'], e)})
+
+    # if state = present and purge_users set delete users which are in members having give access level but not in gitlab_users
+    if state == 'present' and purge_users:
+        uppercase_names_in_gitlab_users_access = []
+        for name in gitlab_users_access:
+            uppercase_names_in_gitlab_users_access.append(name['name'].upper())
+
+        for member in members:
+            if member.access_level in purge_users and member.username.upper() not in uppercase_names_in_gitlab_users_access:
+                try:
+                    if not module.check_mode:
+                        group.remove_user_from_group(member.id, gitlab_group_id)
+                    changed = True
+                    changed_users.append("Successfully removed user '%s', from group. Was not in given list" % member.username)
+                    changed_data.append({'gitlab_user': member.username, 'result': 'CHANGED',
+                                         'msg': "Successfully removed user '%s', from group. Was not in given list" % member.username})
+                except (gitlab.exceptions.GitlabDeleteError) as e:
+                    error = True
+                    changed_users.append("Failed to removed user, '%s', from the group" % gitlab_user['name'])
+                    changed_data.append({'gitlab_user': gitlab_user['name'], 'result': 'FAILED',
+                                         'msg': "Failed to remove user, '%s' from the group: %s" % (gitlab_user['name'], e)})
+
+    if len(gitlab_users_access) == 1 and error:
+        # if single user given and an error occurred return error for list errors will be per user
+        module.fail_json(msg="FAILED: '%s '" % changed_users[0], result_data=changed_data)
+    elif error:
+        module.fail_json(msg='FAILED: At least one given user/permission could not be set', result_data=changed_data)
+
+    module.exit_json(changed=changed, msg='Successfully set memberships', result="\n".join(changed_users), result_data=changed_data)
 
 
 if __name__ == '__main__':

tests/integration/targets/gitlab_group_members/tasks/main.yml

@@ -28,3 +28,46 @@
     gitlab_group: '{{ gitlab_group_name }}'
     gitlab_user: '{{ username }}'
     state: absent
+
+- name: Add a list of Users to A GitLab Group
+  gitlab_group_members:
+    api_url: '{{ gitlab_server_url }}'
+    api_token: '{{ gitlab_api_access_token }}'
+    gitlab_group: '{{ gitlab_group_name }}'
+    gitlab_user: '{{ userlist }}'
+    access_level: '{{ gitlab_access_level }}'
+    state: present
+
+- name: Remove a list of Users to A GitLab Group
+  gitlab_group_members:
+    api_url: '{{ gitlab_server_url }}'
+    api_token: '{{ gitlab_api_access_token }}'
+    gitlab_group: '{{ gitlab_group_name }}'
+    gitlab_user: '{{ userlist }}'
+    state: absent
+
+- name: Add a list of Users with Dedicated Access Levels to A GitLab Group
+  gitlab_group_members:
+    api_url: '{{ gitlab_server_url }}'
+    api_token: '{{ gitlab_api_access_token }}'
+    gitlab_group: '{{ gitlab_group_name }}'
+    gitlab_users_access: '{{ dedicated_access_users }}'
+    state: present
+
+- name: Remove a list of Users with Dedicated Access Levels to A GitLab Group
+  gitlab_group_members:
+    api_url: '{{ gitlab_server_url }}'
+    api_token: '{{ gitlab_api_access_token }}'
+    gitlab_group: '{{ gitlab_group_name }}'
+    gitlab_users_access: '{{ dedicated_access_users }}'
+    state: absent
+
+- name: Add a user, remove all others which might be on this access level
+  gitlab_group_members:
+    api_url: '{{ gitlab_server_url }}'
+    api_token: '{{ gitlab_api_access_token }}'
+    gitlab_group: '{{ gitlab_group_name }}'
+    gitlab_user: '{{ username }}'
+    access_level: '{{ gitlab_access_level }}'
+    pruge_users: '{{ gitlab_access_level }}'
+    state: present

tests/integration/targets/gitlab_group_members/vars/main.yml

@@ -3,3 +3,11 @@ gitlab_api_access_token: 126hngbscx890cv09b
 gitlab_group_name: groupname1
 username: username1
 gitlab_access_level: developer
+userlist:
+  - username1
+  - username2
+dedicated_access_users:
+  - name: username1
+    access_level: "developer"
+  - name: username2
+    access_level: "maintainer"