ansible/community.general
1
0
Fork 0
mirror of https://github.com/ansible-collections/community.general.git synced 2024-09-14 20:13:21 +02:00
Code Releases Activity

cpanm - revamp module (#2218)

Browse source 
* copying from the previous branch

* passing sanity - docs incomplete

* adjusted parameter name

* adjusted unit tests for mode=new

* adjusted integration tests for mode=new

* added 'russoz' to list of maintainers for cpanm

* Update tests/integration/targets/cpanm/tasks/main.yml

* Update tests/integration/targets/cpanm/tasks/main.yml

* ensuring backward compatibility + tests

* added changelog fragment

* version for new parameter and adjusted example

* typo and formatting

* Update plugins/modules/packaging/language/cpanm.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update plugins/modules/packaging/language/cpanm.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* Update plugins/modules/packaging/language/cpanm.py

Co-authored-by: Felix Fontein <felix@fontein.de>

* multiple changes

- some fixes from PR
- supporting tests
- integration is no longer unsupported => destructive, should run on
  apt- and rpm-based systems only

* only run integration tests in redhat-family > v7 or debian-family

Co-authored-by: Felix Fontein <felix@fontein.de>
This commit is contained in:
Alexei Znamensky 2021-04-18 20:55:47 +12:00 committed by GitHub
parent 721589827e
commit ec9c23437c
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
7 changed files with 508 additions and 101 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
.github/BOTMETA.yml vendored
View file

@ -545,7 +545,7 @@ files:
$modules/packaging/language/composer.py:
maintainers: dmtrs resmo
$modules/packaging/language/cpanm.py:
maintainers: fcuny
maintainers: fcuny russoz
$modules/packaging/language/easy_install.py:
maintainers: mattupstate
$modules/packaging/language/gem.py:

5
changelogs/fragments/2218-cpanm-revamp.yml Normal file
View file

@ -0,0 +1,5 @@
minor_changes:
- cpanm - rewritten using ``ModuleHelper`` (https://github.com/ansible-collections/community.general/pull/2218).
- cpanm - honor and install specified version when running in ``new`` mode; that feature is not available in ``compatibility`` mode (https://github.com/ansible-collections/community.general/issues/208).
deprecated_features:
- cpanm - parameter ``system_lib`` deprecated in favor of using ``become`` (https://github.com/ansible-collections/community.general/pull/2218).

242
plugins/modules/packaging/language/cpanm.py
View file

@ -2,6 +2,7 @@
# -*- coding: utf-8 -*-
# (c) 2012, Franck Cuny <franck@lumberjaph.net>
# (c) 2021, Alexei Znamensky <russoz@gmail.com>
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import absolute_import, division, print_function
@ -13,58 +14,91 @@ DOCUMENTATION = '''
module: cpanm
short_description: Manages Perl library dependencies.
description:
- Manage Perl library dependencies.
- Manage Perl library dependencies using cpanminus.
options:
name:
type: str
description:
- The name of the Perl library to install. You may use the "full distribution path", e.g. MIYAGAWA/Plack-0.99_05.tar.gz
aliases: ["pkg"]
- The Perl library to install. Valid values change according to the I(mode), see notes for more details.
- Note that for installing from a local path the parameter I(from_path) should be used.
aliases: [pkg]
from_path:
type: path
description:
- The local directory from where to install
- The local directory or C(tar.gz) file to install from.
notest:
description:
- Do not run unit tests
- Do not run unit tests.
type: bool
default: no
locallib:
description:
- Specify the install base to install modules
- Specify the install base to install modules.
type: path
mirror:
description:
- Specifies the base URL for the CPAN mirror to use
- Specifies the base URL for the CPAN mirror to use.
type: str
mirror_only:
description:
- Use the mirror's index file instead of the CPAN Meta DB
- Use the mirror's index file instead of the CPAN Meta DB.
type: bool
default: no
installdeps:
description:
- Only install dependencies
- Only install dependencies.
type: bool
default: no
version:
description:
- minimum version of perl module to consider acceptable
- Version specification for the perl module. When I(mode) is C(new), C(cpanm) version operators are accepted.
type: str
system_lib:
description:
- Use this if you want to install modules to the system perl include path. You must be root or have "passwordless" sudo for this to work.
- This uses the cpanm commandline option '--sudo', which has nothing to do with ansible privilege escalation.
- Use this if you want to install modules to the system perl include path. You must be root or have "passwordless" sudo for this to work.
- This uses the cpanm commandline option C(--sudo), which has nothing to do with ansible privilege escalation.
- >
This option is not recommended for use and it will be deprecated in the future. If you need to escalate privileges
please consider using any of the multiple mechanisms available in Ansible.
type: bool
default: no
aliases: ['use_sudo']
executable:
description:
- Override the path to the cpanm executable
- Override the path to the cpanm executable.
type: path
mode:
description:
- Controls the module behavior. See notes below for more details.
type: str
choices: [compatibility, new]
default: compatibility
version_added: 3.0.0
name_check:
description:
- When in C(new) mode, this parameter can be used to check if there is a module I(name) installed (at I(version), when specified).
type: str
version_added: 3.0.0
notes:
- Please note that U(http://search.cpan.org/dist/App-cpanminus/bin/cpanm, cpanm) must be installed on the remote host.
author: "Franck Cuny (@fcuny)"
- Please note that U(http://search.cpan.org/dist/App-cpanminus/bin/cpanm, cpanm) must be installed on the remote host.
- "This module now comes with a choice of execution I(mode): C(compatibility) or C(new)."
- "C(compatibility) mode:"
- When using C(compatibility) mode, the module will keep backward compatibility. This is the default mode.
- I(name) must be either a module name or a distribution file.
- >
If the perl module given by I(name) is installed (at the exact I(version) when specified), then nothing happens.
Otherwise, it will be installed using the C(cpanm) executable.
- I(name) cannot be an URL, or a git URL.
- C(cpanm) version specifiers do not work in this mode.
- "C(new) mode:"
- "When using C(new) mode, the module will behave differently"
- >
The I(name) parameter may refer to a module name, a distribution file,
a HTTP URL or a git repository URL as described in C(cpanminus) documentation.
- C(cpanm) version specifiers are recognized.
author:
- "Franck Cuny (@fcuny)"
- "Alexei Znamensky (@russoz)"
'''
EXAMPLES = '''
@ -97,9 +131,9 @@ EXAMPLES = '''
mirror: 'http://cpan.cpantesters.org/'
- name: Install Dancer perl package into the system root path
become: yes
community.general.cpanm:
name: Dancer
system_lib: yes
- name: Install Dancer if it is not already installed OR the installed version is older than version 1.0
community.general.cpanm:
@ -109,105 +143,113 @@ EXAMPLES = '''
import os
from ansible.module_utils.basic import AnsibleModule
from ansible_collections.community.general.plugins.module_utils.module_helper import (
ModuleHelper, CmdMixin, ArgFormat, ModuleHelperException
)
def _is_package_installed(module, name, locallib, cpanm, version):
cmd = ""
if locallib:
os.environ["PERL5LIB"] = "%s/lib/perl5" % locallib
cmd = "%s perl -e ' use %s" % (cmd, name)
if version:
cmd = "%s %s;'" % (cmd, version)
else:
cmd = "%s;'" % cmd
res, stdout, stderr = module.run_command(cmd, check_rc=False)
return res == 0
class CPANMinus(CmdMixin, ModuleHelper):
output_params = ['name', 'version']
module = dict(
argument_spec=dict(
name=dict(type='str', aliases=['pkg']),
version=dict(type='str'),
from_path=dict(type='path'),
notest=dict(type='bool', default=False),
locallib=dict(type='path'),
mirror=dict(type='str'),
mirror_only=dict(type='bool', default=False),
installdeps=dict(type='bool', default=False),
system_lib=dict(type='bool', default=False, aliases=['use_sudo'],
removed_in_version="4.0.0", removed_from_collection="community.general"),
executable=dict(type='path'),
mode=dict(type='str', choices=['compatibility', 'new'], default='compatibility'),
name_check=dict(type='str')
),
required_one_of=[('name', 'from_path')],
)
command = 'cpanm'
command_args_formats = dict(
notest=dict(fmt="--notest", style=ArgFormat.BOOLEAN),
locallib=dict(fmt=('--local-lib', '{0}'),),
mirror=dict(fmt=('--mirror', '{0}'),),
mirror_only=dict(fmt="--mirror-only", style=ArgFormat.BOOLEAN),
installdeps=dict(fmt="--installdeps", style=ArgFormat.BOOLEAN),
system_lib=dict(fmt="--sudo", style=ArgFormat.BOOLEAN),
)
check_rc = True
def _build_cmd_line(name, from_path, notest, locallib, mirror, mirror_only, installdeps, cpanm, use_sudo):
# this code should use "%s" like everything else and just return early but not fixing all of it now.
# don't copy stuff like this
if from_path:
cmd = cpanm + " " + from_path
else:
cmd = cpanm + " " + name
def __init_module__(self):
v = self.vars
if v.mode == "compatibility":
if v.name_check:
raise ModuleHelperException("Parameter name_check can only be used with mode=new")
else:
if v.name and v.from_path:
raise ModuleHelperException("Parameters 'name' and 'from_path' are mutually exclusive when 'mode=new'")
if v.system_lib:
raise ModuleHelperException("Parameter 'system_lib' is invalid when 'mode=new'")
if notest is True:
cmd = cmd + " -n"
self.command = self.module.get_bin_path(v.executable if v.executable else self.command)
self.vars.set("binary", self.command)
if locallib is not None:
cmd = cmd + " -l " + locallib
def _is_package_installed(self, name, locallib, version):
if name is None or name.endswith('.tar.gz'):
return False
version = "" if version is None else " " + version
if mirror is not None:
cmd = cmd + " --mirror " + mirror
env = {"PERL5LIB": "%s/lib/perl5" % locallib} if locallib else {}
cmd = ['perl', '-le', 'use %s%s;' % (name, version)]
rc, out, err = self.module.run_command(cmd, check_rc=False, environ_update=env)
if mirror_only is True:
cmd = cmd + " --mirror-only"
return rc == 0
if installdeps is True:
cmd = cmd + " --installdeps"
@staticmethod
def sanitize_pkg_spec_version(pkg_spec, version):
if version is None:
return pkg_spec
if pkg_spec.endswith('.tar.gz'):
raise ModuleHelperException(msg="parameter 'version' must not be used when installing from a file")
if os.path.isdir(pkg_spec):
raise ModuleHelperException(msg="parameter 'version' must not be used when installing from a directory")
if pkg_spec.endswith('.git'):
if version.startswith('~'):
raise ModuleHelperException(msg="operator '~' not allowed in version parameter when installing from git repository")
version = version if version.startswith('@') else '@' + version
elif version[0] not in ('@', '~'):
version = '~' + version
return pkg_spec + version
if use_sudo is True:
cmd = cmd + " --sudo"
def __run__(self):
v = self.vars
pkg_param = 'from_path' if v.from_path else 'name'
return cmd
if v.mode == 'compatibility':
if self._is_package_installed(v.name, v.locallib, v.version):
return
pkg_spec = v[pkg_param]
self.changed = self.run_command(
params=['notest', 'locallib', 'mirror', 'mirror_only', 'installdeps', 'system_lib', {'name': pkg_spec}],
)
else:
installed = self._is_package_installed(v.name_check, v.locallib, v.version) if v.name_check else False
if installed:
return
pkg_spec = self.sanitize_pkg_spec_version(v[pkg_param], v.version)
self.changed = self.run_command(
params=['notest', 'locallib', 'mirror', 'mirror_only', 'installdeps', {'name': pkg_spec}],
)
def _get_cpanm_path(module):
if module.params['executable']:
result = module.params['executable']
else:
result = module.get_bin_path('cpanm', True)
return result
def process_command_output(self, rc, out, err):
if self.vars.mode == "compatibility" and rc != 0:
raise ModuleHelperException(msg=err, cmd=self.vars.cmd_args)
return 'is up to date' not in err and 'is up to date' not in out
def main():
arg_spec = dict(
name=dict(default=None, required=False, aliases=['pkg']),
from_path=dict(default=None, required=False, type='path'),
notest=dict(default=False, type='bool'),
locallib=dict(default=None, required=False, type='path'),
mirror=dict(default=None, required=False),
mirror_only=dict(default=False, type='bool'),
installdeps=dict(default=False, type='bool'),
system_lib=dict(default=False, type='bool', aliases=['use_sudo']),
version=dict(default=None, required=False),
executable=dict(required=False, type='path'),
)
module = AnsibleModule(
argument_spec=arg_spec,
required_one_of=[['name', 'from_path']],
)
cpanm = _get_cpanm_path(module)
name = module.params['name']
from_path = module.params['from_path']
notest = module.boolean(module.params.get('notest', False))
locallib = module.params['locallib']
mirror = module.params['mirror']
mirror_only = module.params['mirror_only']
installdeps = module.params['installdeps']
use_sudo = module.params['system_lib']
version = module.params['version']
changed = False
installed = _is_package_installed(module, name, locallib, cpanm, version)
if not installed:
cmd = _build_cmd_line(name, from_path, notest, locallib, mirror, mirror_only, installdeps, cpanm, use_sudo)
rc_cpanm, out_cpanm, err_cpanm = module.run_command(cmd, check_rc=False)
if rc_cpanm != 0:
module.fail_json(msg=err_cpanm, cmd=cmd)
if (err_cpanm.find('is up to date') == -1 and out_cpanm.find('is up to date') == -1):
changed = True
module.exit_json(changed=changed, binary=cpanm, name=name)
cpanm = CPANMinus()
cpanm.run()
if __name__ == '__main__':

6
tests/integration/targets/cpanm/aliases Normal file
View file

@ -0,0 +1,6 @@
shippable/posix/group3
destructive
skip/macos
skip/osx
skip/freebsd
skip/aix

2
tests/integration/targets/cpanm/meta/main.yml Normal file
View file

@ -0,0 +1,2 @@
dependencies:
- setup_pkg_mgr

64
tests/integration/targets/cpanm/tasks/main.yml Normal file
View file

@ -0,0 +1,64 @@
# (c) 2020, Berkhan Berkdemir
# (c) 2021, Alexei Znamensky
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
- name: bail out for non-supported platforms
meta: end_play
when:
- (ansible_os_family != "RedHat" or ansible_distribution_major_version|int < 7)
- ansible_os_family != "Debian"
- name: install perl development package for Red Hat family
package:
name:
- perl-devel
- perl-App-cpanminus
state: present
become: yes
when: ansible_os_family == "RedHat"
- name: install perl development package for Debian family
package:
name:
- cpanminus
state: present
become: yes
when: ansible_os_family == "Debian"
- name: install a Perl package
cpanm:
name: JSON
notest: yes
register: install_perl_package_result
- name: assert package is installed
assert:
that:
- install_perl_package_result is changed
- install_perl_package_result is not failed
- name: install same Perl package
cpanm:
name: JSON
notest: yes
register: install_same_perl_package_result
- name: assert same package is installed
assert:
that:
- install_same_perl_package_result is not changed
- install_same_perl_package_result is not failed
- name: install a Perl package with version operator
cpanm:
name: JSON
version: "@4.01"
notest: yes
mode: new
register: install_perl_package_with_version_op_result
- name: assert package with version operator is installed
assert:
that:
- install_perl_package_with_version_op_result is changed
- install_perl_package_with_version_op_result is not failed

288
tests/unit/plugins/modules/packaging/language/test_cpanm.py Normal file
View file

@ -0,0 +1,288 @@
# Author: Alexei Znamensky (russoz@gmail.com)
# Largely adapted from test_redhat_subscription by
# Jiri Hnidek (jhnidek@redhat.com)
#
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
import json
from ansible_collections.community.general.plugins.modules.packaging.language import cpanm
import pytest
TESTED_MODULE = cpanm.__name__
@pytest.fixture
def patch_cpanm(mocker):
"""
Function used for mocking some parts of redhat_subscribtion module
"""
mocker.patch('ansible_collections.community.general.plugins.module_utils.module_helper.AnsibleModule.get_bin_path',
return_value='/testbin/cpanm')
TEST_CASES = [
[
{'name': 'Dancer'},
{
'id': 'install_dancer_compatibility',
'run_command.calls': [
(
['perl', '-le', 'use Dancer;'],
{'environ_update': {}, 'check_rc': False},
(2, '', 'error, not installed',), # output rc, out, err
),
(
['/testbin/cpanm', 'Dancer'],
{'environ_update': {'LANGUAGE': 'C'}, 'check_rc': True},
(0, '', '',), # output rc, out, err
),
],
'changed': True,
}
],
[
{'name': 'Dancer'},
{
'id': 'install_dancer_already_installed_compatibility',
'run_command.calls': [
(
['perl', '-le', 'use Dancer;'],
{'environ_update': {}, 'check_rc': False},
(0, '', '',), # output rc, out, err
),
],
'changed': False,
}
],
[
{'name': 'Dancer', 'mode': 'new'},
{
'id': 'install_dancer',
'run_command.calls': [(
['/testbin/cpanm', 'Dancer'],
{'environ_update': {'LANGUAGE': 'C'}, 'check_rc': True},
(0, '', '',), # output rc, out, err
)],
'changed': True,
}
],
[
{'name': 'MIYAGAWA/Plack-0.99_05.tar.gz'},
{
'id': 'install_distribution_file_compatibility',
'run_command.calls': [(
['/testbin/cpanm', 'MIYAGAWA/Plack-0.99_05.tar.gz'],
{'environ_update': {'LANGUAGE': 'C'}, 'check_rc': True},
(0, '', '',), # output rc, out, err
)],
'changed': True,
}
],
[
{'name': 'MIYAGAWA/Plack-0.99_05.tar.gz', 'mode': 'new'},
{
'id': 'install_distribution_file',
'run_command.calls': [(
['/testbin/cpanm', 'MIYAGAWA/Plack-0.99_05.tar.gz'],
{'environ_update': {'LANGUAGE': 'C'}, 'check_rc': True},
(0, '', '',), # output rc, out, err
)],
'changed': True,
}
],
[
{'name': 'Dancer', 'locallib': '/srv/webapps/my_app/extlib', 'mode': 'new'},
{
'id': 'install_into_locallib',
'run_command.calls': [(
['/testbin/cpanm', '--local-lib', '/srv/webapps/my_app/extlib', 'Dancer'],
{'environ_update': {'LANGUAGE': 'C'}, 'check_rc': True},
(0, '', '',), # output rc, out, err
)],
'changed': True,
}
],
[
{'from_path': '/srv/webapps/my_app/src/', 'mode': 'new'},
{
'id': 'install_from_local_directory',
'run_command.calls': [(
['/testbin/cpanm', '/srv/webapps/my_app/src/'],
{'environ_update': {'LANGUAGE': 'C'}, 'check_rc': True},
(0, '', '',), # output rc, out, err
)],
'changed': True,
}
],
[
{'name': 'Dancer', 'locallib': '/srv/webapps/my_app/extlib', 'notest': True, 'mode': 'new'},
{
'id': 'install_into_locallib_no_unit_testing',
'run_command.calls': [(
['/testbin/cpanm', '--notest', '--local-lib', '/srv/webapps/my_app/extlib', 'Dancer'],
{'environ_update': {'LANGUAGE': 'C'}, 'check_rc': True},
(0, '', '',), # output rc, out, err
)],
'changed': True,
}
],
[
{'name': 'Dancer', 'mirror': 'http://cpan.cpantesters.org/', 'mode': 'new'},
{
'id': 'install_from_mirror',
'run_command.calls': [(
['/testbin/cpanm', '--mirror', 'http://cpan.cpantesters.org/', 'Dancer'],
{'environ_update': {'LANGUAGE': 'C'}, 'check_rc': True},
(0, '', '',), # output rc, out, err
)],
'changed': True,
}
],
[
{'name': 'Dancer', 'system_lib': True, 'mode': 'new'},
{
'id': 'install_into_system_lib',
'run_command.calls': [],
'changed': False,
'failed': True,
}
],
[
{'name': 'Dancer', 'version': '1.0', 'mode': 'new'},
{
'id': 'install_minversion_implicit',
'run_command.calls': [(
['/testbin/cpanm', 'Dancer~1.0'],
{'environ_update': {'LANGUAGE': 'C'}, 'check_rc': True},
(0, '', '',), # output rc, out, err
)],
'changed': True,
}
],
[
{'name': 'Dancer', 'version': '~1.5', 'mode': 'new'},
{
'id': 'install_minversion_explicit',
'run_command.calls': [(
['/testbin/cpanm', 'Dancer~1.5'],
{'environ_update': {'LANGUAGE': 'C'}, 'check_rc': True},
(0, '', '',), # output rc, out, err
)],
'changed': True,
}
],
[
{'name': 'Dancer', 'version': '@1.7', 'mode': 'new'},
{
'id': 'install_specific_version',
'run_command.calls': [(
['/testbin/cpanm', 'Dancer@1.7'],
{'environ_update': {'LANGUAGE': 'C'}, 'check_rc': True},
(0, '', '',), # output rc, out, err
)],
'changed': True,
'failed': False,
}
],
[
{'name': 'MIYAGAWA/Plack-0.99_05.tar.gz', 'version': '@1.7', 'mode': 'new'},
{
'id': 'install_specific_version_from_file_error',
'run_command.calls': [],
'changed': False,
'failed': True,
'msg': "parameter 'version' must not be used when installing from a file",
}
],
[
{'from_path': '~/', 'version': '@1.7', 'mode': 'new'},
{
'id': 'install_specific_version_from_directory_error',
'run_command.calls': [],
'changed': False,
'failed': True,
'msg': "parameter 'version' must not be used when installing from a directory",
}
],
[
{'name': 'git://github.com/plack/Plack.git', 'version': '@1.7', 'mode': 'new'},
{
'id': 'install_specific_version_from_git_url_explicit',
'run_command.calls': [(
['/testbin/cpanm', 'git://github.com/plack/Plack.git@1.7'],
{'environ_update': {'LANGUAGE': 'C'}, 'check_rc': True},
(0, '', '',), # output rc, out, err
)],
'changed': True,
'failed': False,
}
],
[
{'name': 'git://github.com/plack/Plack.git', 'version': '2.5', 'mode': 'new'},
{
'id': 'install_specific_version_from_git_url_implicit',
'run_command.calls': [(
['/testbin/cpanm', 'git://github.com/plack/Plack.git@2.5'],
{'environ_update': {'LANGUAGE': 'C'}, 'check_rc': True},
(0, '', '',), # output rc, out, err
)],
'changed': True,
'failed': False,
}
],
[
{'name': 'git://github.com/plack/Plack.git', 'version': '~2.5', 'mode': 'new'},
{
'id': 'install_version_operator_from_git_url_error',
'run_command.calls': [],
'changed': False,
'failed': True,
'msg': "operator '~' not allowed in version parameter when installing from git repository",
}
],
]
TEST_CASES_IDS = [item[1]['id'] for item in TEST_CASES]
@pytest.mark.parametrize('patch_ansible_module, testcase',
TEST_CASES,
ids=TEST_CASES_IDS,
indirect=['patch_ansible_module'])
@pytest.mark.usefixtures('patch_ansible_module')
def test_cpanm(mocker, capfd, patch_cpanm, testcase):
"""
Run unit tests for test cases listen in TEST_CASES
"""
# Mock function used for running commands first
call_results = [item[2] for item in testcase['run_command.calls']]
mock_run_command = mocker.patch(
'ansible_collections.community.general.plugins.module_utils.module_helper.AnsibleModule.run_command',
side_effect=call_results)
# Try to run test case
with pytest.raises(SystemExit):
cpanm.main()
out, err = capfd.readouterr()
results = json.loads(out)
print("results =\n%s" % results)
assert mock_run_command.call_count == len(testcase['run_command.calls'])
if mock_run_command.call_count:
call_args_list = [(item[0][0], item[1]) for item in mock_run_command.call_args_list]
expected_call_args_list = [(item[0], item[1]) for item in testcase['run_command.calls']]
print("call args list =\n%s" % call_args_list)
print("expected args list =\n%s" % expected_call_args_list)
assert call_args_list == expected_call_args_list
assert results.get('changed', False) == testcase['changed']
if 'failed' in testcase:
assert results.get('failed', False) == testcase['failed']
if 'msg' in testcase:
assert results.get('msg', '') == testcase['msg']