fixed Python 3 keys() usage (#1861) (#1890)

* fixed python3 keys()

* added changelog fragment

* Update plugins/modules/cloud/spotinst/spotinst_aws_elastigroup.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update plugins/cache/redis.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* rolledback redis.py per PR

* Update plugins/modules/monitoring/sensu/sensu_check.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* removed unnecessary ignore lines

* adding memcached and one case in redis is indeed necessary

* Update changelogs/fragments/1861-python3-keys.yml

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update changelogs/fragments/1861-python3-keys.yml

* Update changelogs/fragments/1861-python3-keys.yml

Co-authored-by: Felix Fontein <felix@fontein.de>

Co-authored-by: Felix Fontein <felix@fontein.de>
(cherry picked from commit 434f383ae9)

Co-authored-by: Alexei Znamensky <103110+russoz@users.noreply.github.com>

changelogs/fragments/1861-python3-keys.yml

@@ -0,0 +1,22 @@
+bugfixes:
+  - redis cache plugin - wrapped usages of ``keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - memcached cache plugin - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - diy callback plugin - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - selective callback plugin - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - chef_databag lookup plugin - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - net_tools.nios.api module_utils - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - utm_utils module_utils - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - lxc_container - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - lxd_container - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - oneandone_monitoring_policy - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - oci_vcn - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - spotinst_aws_elastigroup - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - sensu_check - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - redhat_subscription - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - idrac_redfish_command - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - idrac_redfish_config - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - idrac_redfish_info - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - redfish_command - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - redfish_config - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - vdo - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).
+  - nsot inventory script - wrapped usages of ``dict.keys()`` in ``list()`` for Python 3 compatibility (https://github.com/ansible-collections/community.general/pull/1861).

plugins/cache/memcached.py

@@ -161,7 +161,7 @@ class CacheModuleKeys(MutableSet):
         self._cache.set(self.PREFIX, self._keyset)
 
     def remove_by_timerange(self, s_min, s_max):
-        for k in self._keyset.keys():
+        for k in list(self._keyset.keys()):
             t = self._keyset[k]
             if s_min < t < s_max:
                 del self._keyset[k]

plugins/cache/redis.py

@@ -216,14 +216,12 @@ class CacheModule(BaseCacheModule):
         self._db.zrem(self._keys_set, key)
 
     def flush(self):
-        for key in self.keys():
+        for key in list(self.keys()):
             self.delete(key)
 
     def copy(self):
         # TODO: there is probably a better way to do this in redis
-        ret = dict()
-        for key in self.keys():
-            ret[key] = self.get(key)
+        ret = dict([(k, self.get(k)) for k in self.keys()])
         return ret
 
     def __getstate__(self):

plugins/callback/diy.py

@@ -1013,7 +1013,7 @@ class CallbackModule(Default):
             for attr in _stats_attributes:
                 _ret[self.DIY_NS]['stats'].update({attr: _get_value(obj=stats, attr=attr)})
 
-        _ret[self.DIY_NS].update({'top_level_var_names': _ret.keys()})
+        _ret[self.DIY_NS].update({'top_level_var_names': list(_ret.keys())})
 
         return _ret
 

plugins/callback/selective.py

@@ -67,7 +67,7 @@ COLORS = {
 
 def dict_diff(prv, nxt):
     """Return a dict of keys that differ with another config object."""
-    keys = set(prv.keys() + nxt.keys())
+    keys = set(list(prv.keys()) + list(nxt.keys()))
     result = {}
     for k in keys:
         if prv.get(k) != nxt.get(k):

plugins/lookup/chef_databag.py

@@ -81,7 +81,7 @@ class LookupModule(LookupBase):
                 )
         if args:
             raise AnsibleError(
-                "unrecognized arguments to with_sequence: %r" % args.keys()
+                "unrecognized arguments to with_sequence: %r" % list(args.keys())
             )
 
     def run(self, terms, variables=None, **kwargs):

plugins/module_utils/net_tools/nios/api.py

@@ -490,12 +490,12 @@ class WapiModule(WapiBase):
                 else:
                     test_obj_filter = dict([('name', old_name)])
                 # get the object reference
-                ib_obj = self.get_object(ib_obj_type, test_obj_filter, return_fields=ib_spec.keys())
+                ib_obj = self.get_object(ib_obj_type, test_obj_filter, return_fields=list(ib_spec.keys()))
                 if ib_obj:
                     obj_filter['name'] = new_name
                 else:
                     test_obj_filter['name'] = new_name
-                    ib_obj = self.get_object(ib_obj_type, test_obj_filter, return_fields=ib_spec.keys())
+                    ib_obj = self.get_object(ib_obj_type, test_obj_filter, return_fields=list(ib_spec.keys()))
                 update = True
                 return ib_obj, update, new_name
             if (ib_obj_type == NIOS_HOST_RECORD):
@@ -529,7 +529,7 @@ class WapiModule(WapiBase):
             # check if test_obj_filter is empty copy passed obj_filter
             else:
                 test_obj_filter = obj_filter
-            ib_obj = self.get_object(ib_obj_type, test_obj_filter.copy(), return_fields=ib_spec.keys())
+            ib_obj = self.get_object(ib_obj_type, test_obj_filter.copy(), return_fields=list(ib_spec.keys()))
         elif (ib_obj_type == NIOS_A_RECORD):
             # resolves issue where multiple a_records with same name and different IP address
             test_obj_filter = obj_filter
@@ -539,7 +539,7 @@ class WapiModule(WapiBase):
             except TypeError:
                 ipaddr = obj_filter['ipv4addr']
             test_obj_filter['ipv4addr'] = ipaddr
-            ib_obj = self.get_object(ib_obj_type, test_obj_filter.copy(), return_fields=ib_spec.keys())
+            ib_obj = self.get_object(ib_obj_type, test_obj_filter.copy(), return_fields=list(ib_spec.keys()))
         elif (ib_obj_type == NIOS_TXT_RECORD):
             # resolves issue where multiple txt_records with same name and different text
             test_obj_filter = obj_filter
@@ -549,12 +549,12 @@ class WapiModule(WapiBase):
             except TypeError:
                 txt = obj_filter['text']
             test_obj_filter['text'] = txt
-            ib_obj = self.get_object(ib_obj_type, test_obj_filter.copy(), return_fields=ib_spec.keys())
+            ib_obj = self.get_object(ib_obj_type, test_obj_filter.copy(), return_fields=list(ib_spec.keys()))
         elif (ib_obj_type == NIOS_ZONE):
             # del key 'restart_if_needed' as nios_zone get_object fails with the key present
             temp = ib_spec['restart_if_needed']
             del ib_spec['restart_if_needed']
-            ib_obj = self.get_object(ib_obj_type, obj_filter.copy(), return_fields=ib_spec.keys())
+            ib_obj = self.get_object(ib_obj_type, obj_filter.copy(), return_fields=list(ib_spec.keys()))
             # reinstate restart_if_needed if ib_obj is none, meaning there's no existing nios_zone ref
             if not ib_obj:
                 ib_spec['restart_if_needed'] = temp
@@ -562,12 +562,12 @@ class WapiModule(WapiBase):
             # del key 'create_token' as nios_member get_object fails with the key present
             temp = ib_spec['create_token']
             del ib_spec['create_token']
-            ib_obj = self.get_object(ib_obj_type, obj_filter.copy(), return_fields=ib_spec.keys())
+            ib_obj = self.get_object(ib_obj_type, obj_filter.copy(), return_fields=list(ib_spec.keys()))
             if temp:
                 # reinstate 'create_token' key
                 ib_spec['create_token'] = temp
         else:
-            ib_obj = self.get_object(ib_obj_type, obj_filter.copy(), return_fields=ib_spec.keys())
+            ib_obj = self.get_object(ib_obj_type, obj_filter.copy(), return_fields=list(ib_spec.keys()))
         return ib_obj, update, new_name
 
     def on_update(self, proposed_object, ib_spec):

plugins/module_utils/utm_utils.py

@@ -84,7 +84,7 @@ class UTM:
             raise UTMModuleConfigurationError(
                 "The keys " + to_native(
                     self.change_relevant_keys) + " to check are not in the modules keys:\n" + to_native(
-                    module.params.keys()))
+                    list(module.params.keys())))
 
     def execute(self):
         try:

plugins/modules/cloud/lxc/lxc_container.py

@@ -1662,7 +1662,7 @@ def main():
             ),
             backing_store=dict(
                 type='str',
-                choices=LXC_BACKING_STORE.keys(),
+                choices=list(LXC_BACKING_STORE.keys()),
                 default='dir'
             ),
             template_options=dict(
@@ -1699,7 +1699,7 @@ def main():
                 type='path'
             ),
             state=dict(
-                choices=LXC_ANSIBLE_STATES.keys(),
+                choices=list(LXC_ANSIBLE_STATES.keys()),
                 default='started'
             ),
             container_command=dict(
@@ -1733,7 +1733,7 @@ def main():
                 type='path',
             ),
             archive_compression=dict(
-                choices=LXC_COMPRESSION_MAP.keys(),
+                choices=list(LXC_COMPRESSION_MAP.keys()),
                 default='gzip'
             )
         ),

plugins/modules/cloud/lxd/lxd_container.py

@@ -665,7 +665,7 @@ def main():
                 type='dict',
             ),
             state=dict(
-                choices=LXD_ANSIBLE_STATES.keys(),
+                choices=list(LXD_ANSIBLE_STATES.keys()),
                 default='started'
             ),
             target=dict(

plugins/modules/cloud/oneandone/oneandone_monitoring_policy.py

@@ -695,15 +695,15 @@ def update_monitoring_policy(module, oneandone_conn):
             threshold_entities = ['cpu', 'ram', 'disk', 'internal_ping', 'transfer']
 
             _thresholds = []
-            for treshold in thresholds:
-                key = treshold.keys()[0]
+            for threshold in thresholds:
+                key = list(threshold.keys())[0]
                 if key in threshold_entities:
                     _threshold = oneandone.client.Threshold(
                         entity=key,
-                        warning_value=treshold[key]['warning']['value'],
-                        warning_alert=str(treshold[key]['warning']['alert']).lower(),
-                        critical_value=treshold[key]['critical']['value'],
-                        critical_alert=str(treshold[key]['critical']['alert']).lower())
+                        warning_value=threshold[key]['warning']['value'],
+                        warning_alert=str(threshold[key]['warning']['alert']).lower(),
+                        critical_value=threshold[key]['critical']['value'],
+                        critical_alert=str(threshold[key]['critical']['alert']).lower())
                     _thresholds.append(_threshold)
 
         if name or description or email or thresholds:
@@ -864,15 +864,15 @@ def create_monitoring_policy(module, oneandone_conn):
         threshold_entities = ['cpu', 'ram', 'disk', 'internal_ping', 'transfer']
 
         _thresholds = []
-        for treshold in thresholds:
-            key = treshold.keys()[0]
+        for threshold in thresholds:
+            key = list(threshold.keys())[0]
             if key in threshold_entities:
                 _threshold = oneandone.client.Threshold(
                     entity=key,
-                    warning_value=treshold[key]['warning']['value'],
-                    warning_alert=str(treshold[key]['warning']['alert']).lower(),
-                    critical_value=treshold[key]['critical']['value'],
-                    critical_alert=str(treshold[key]['critical']['alert']).lower())
+                    warning_value=threshold[key]['warning']['value'],
+                    warning_alert=str(threshold[key]['warning']['alert']).lower(),
+                    critical_value=threshold[key]['critical']['value'],
+                    critical_alert=str(threshold[key]['critical']['alert']).lower())
                 _thresholds.append(_threshold)
 
         _ports = []

plugins/modules/cloud/oracle/oci_vcn.py

@@ -128,7 +128,7 @@ def update_vcn(virtual_network_client, module):
         primitive_params_update=["vcn_id"],
         kwargs_non_primitive_update={UpdateVcnDetails: "update_vcn_details"},
         module=module,
-        update_attributes=UpdateVcnDetails().attribute_map.keys(),
+        update_attributes=list(UpdateVcnDetails().attribute_map.keys()),
     )
     return result
 

plugins/modules/cloud/spotinst/spotinst_aws_elastigroup.py

@@ -1305,10 +1305,8 @@ def expand_tags(eg_launchspec, tags):
 
         for tag in tags:
             eg_tag = spotinst.aws_elastigroup.Tag()
-            if tag.keys():
-                eg_tag.tag_key = tag.keys()[0]
-            if tag.values():
-                eg_tag.tag_value = tag.values()[0]
+            if tag:
+                eg_tag.tag_key, eg_tag.tag_value = list(tag.items())[0]
 
             eg_tags.append(eg_tag)
 

plugins/modules/packaging/os/redhat_subscription.py

@@ -596,7 +596,7 @@ class Rhsm(RegistrationBase):
 
         if missing_pools or serials:
             changed = True
-        return {'changed': changed, 'subscribed_pool_ids': missing_pools.keys(),
+        return {'changed': changed, 'subscribed_pool_ids': list(missing_pools.keys()),
                 'unsubscribed_serials': serials}
 
     def sync_syspurpose(self):

plugins/modules/remote_management/redfish/idrac_redfish_command.py

@@ -165,7 +165,7 @@ def main():
 
     # Check that Category is valid
     if category not in CATEGORY_COMMANDS_ALL:
-        module.fail_json(msg=to_native("Invalid Category '%s'. Valid Categories = %s" % (category, CATEGORY_COMMANDS_ALL.keys())))
+        module.fail_json(msg=to_native("Invalid Category '%s'. Valid Categories = %s" % (category, list(CATEGORY_COMMANDS_ALL.keys()))))
 
     # Check that all commands are valid
     for cmd in command_list:

plugins/modules/remote_management/redfish/idrac_redfish_config.py

@@ -279,7 +279,7 @@ def main():
 
     # Check that Category is valid
     if category not in CATEGORY_COMMANDS_ALL:
-        module.fail_json(msg=to_native("Invalid Category '%s'. Valid Categories = %s" % (category, CATEGORY_COMMANDS_ALL.keys())))
+        module.fail_json(msg=to_native("Invalid Category '%s'. Valid Categories = %s" % (category, list(CATEGORY_COMMANDS_ALL.keys()))))
 
     # Check that all commands are valid
     for cmd in command_list:

plugins/modules/remote_management/redfish/idrac_redfish_info.py

@@ -202,7 +202,7 @@ def main():
 
     # Check that Category is valid
     if category not in CATEGORY_COMMANDS_ALL:
-        module.fail_json(msg=to_native("Invalid Category '%s'. Valid Categories = %s" % (category, CATEGORY_COMMANDS_ALL.keys())))
+        module.fail_json(msg=to_native("Invalid Category '%s'. Valid Categories = %s" % (category, list(CATEGORY_COMMANDS_ALL.keys()))))
 
     # Check that all commands are valid
     for cmd in command_list:

plugins/modules/remote_management/redfish/redfish_command.py

@@ -639,7 +639,7 @@ def main():
 
     # Check that Category is valid
     if category not in CATEGORY_COMMANDS_ALL:
-        module.fail_json(msg=to_native("Invalid Category '%s'. Valid Categories = %s" % (category, CATEGORY_COMMANDS_ALL.keys())))
+        module.fail_json(msg=to_native("Invalid Category '%s'. Valid Categories = %s" % (category, list(CATEGORY_COMMANDS_ALL.keys()))))
 
     # Check that all commands are valid
     for cmd in command_list:

plugins/modules/remote_management/redfish/redfish_config.py

@@ -288,7 +288,7 @@ def main():
 
     # Check that Category is valid
     if category not in CATEGORY_COMMANDS_ALL:
-        module.fail_json(msg=to_native("Invalid Category '%s'. Valid Categories = %s" % (category, CATEGORY_COMMANDS_ALL.keys())))
+        module.fail_json(msg=to_native("Invalid Category '%s'. Valid Categories = %s" % (category, list(CATEGORY_COMMANDS_ALL.keys()))))
 
     # Check that all commands are valid
     for cmd in command_list:

plugins/modules/system/vdo.py

@@ -327,7 +327,7 @@ def inventory_vdos(module, vdocmd):
     vdoyamls = vdostatusyaml['VDOs']
 
     if vdoyamls is not None:
-        vdolist = vdoyamls.keys()
+        vdolist = list(vdoyamls.keys())
 
     return vdolist
 

scripts/inventory/nsot.py

@@ -181,7 +181,7 @@ class NSoTInventory(object):
                     sys.exit('%s\n' % e)
         else:  # Use defaults if env var missing
             self._config_default()
-        self.groups = self.config.keys()
+        self.groups = list(self.config.keys())
         self.client = get_api_client()
         self._meta = {'hostvars': dict()}
 

tests/sanity/ignore-2.10.txt

@@ -62,9 +62,7 @@ plugins/modules/cloud/linode/linode.py validate-modules:parameter-type-not-in-do
 plugins/modules/cloud/linode/linode.py validate-modules:undocumented-parameter
 plugins/modules/cloud/linode/linode_v4.py validate-modules:parameter-list-no-elements
 plugins/modules/cloud/lxc/lxc_container.py use-argspec-type-path
-plugins/modules/cloud/lxc/lxc_container.py validate-modules:invalid-ansiblemodule-schema
 plugins/modules/cloud/lxc/lxc_container.py validate-modules:use-run-command-not-popen
-plugins/modules/cloud/lxd/lxd_container.py validate-modules:invalid-ansiblemodule-schema
 plugins/modules/cloud/misc/rhevm.py validate-modules:parameter-state-invalid-choice
 plugins/modules/cloud/oneandone/oneandone_firewall_policy.py validate-modules:parameter-list-no-elements
 plugins/modules/cloud/oneandone/oneandone_load_balancer.py validate-modules:parameter-list-no-elements

tests/sanity/ignore-2.11.txt

@@ -50,9 +50,7 @@ plugins/modules/cloud/linode/linode.py validate-modules:parameter-type-not-in-do
 plugins/modules/cloud/linode/linode.py validate-modules:undocumented-parameter
 plugins/modules/cloud/linode/linode_v4.py validate-modules:parameter-list-no-elements
 plugins/modules/cloud/lxc/lxc_container.py use-argspec-type-path
-plugins/modules/cloud/lxc/lxc_container.py validate-modules:invalid-ansiblemodule-schema
 plugins/modules/cloud/lxc/lxc_container.py validate-modules:use-run-command-not-popen
-plugins/modules/cloud/lxd/lxd_container.py validate-modules:invalid-ansiblemodule-schema
 plugins/modules/cloud/misc/rhevm.py validate-modules:parameter-state-invalid-choice
 plugins/modules/cloud/oneandone/oneandone_firewall_policy.py validate-modules:parameter-list-no-elements
 plugins/modules/cloud/oneandone/oneandone_load_balancer.py validate-modules:parameter-list-no-elements