ISSUE 25470 - junit report failure on changes

This commit provides an environment option to change the behaviour so
that it's possible to declare any changes shoudl be considered a junit
failure.

This is useful when carrying out idempotent testing to ensure that
multiple runs are safe and any changes should be considered a test
failure.

In a CI test of an ansible role the practice would be to run the role
once without this to configure the test system, and tehn to run a second
time including this environment vairable so that the CI engine
processing the junit report recognise any changes to be a test fail.

CHANGELOG.md

@@ -115,6 +115,7 @@ Ansible Changes By Release
 * Update ipaddr Jinja filters to replace existing non RFC compliant ones. Added additional filters for easier use
   of handling IP addresses. (PR# 26566)
 * datetime filter updated to use default format of datetime.datetime (ISO8601)
+* The junit plugin now has an option to report a junit test failure on changes for idempotent testing.
 
 #### New Callbacks:
 - full_skip

lib/ansible/plugins/callback/junit.py

@@ -58,6 +58,8 @@ class CallbackModule(CallbackBase):
                                      Default: ~/.ansible.log
         JUNIT_TASK_CLASS (optional): Configure the output to be one class per yaml file
                                      Default: False
+        JUNIT_FAIL_ON_CHANGE (optional): Consider any tasks reporting "changed" as a junit test failure
+                                     Default: False
 
     Requires:
         junit_xml
@@ -74,6 +76,7 @@ class CallbackModule(CallbackBase):
 
         self._output_dir = os.getenv('JUNIT_OUTPUT_DIR', os.path.expanduser('~/.ansible.log'))
         self._task_class = os.getenv('JUNIT_TASK_CLASS', 'False').lower()
+        self._fail_on_change = os.getenv('JUNIT_FAIL_ON_CHANGE', 'False').lower()
         self._playbook_path = None
         self._playbook_name = None
         self._play_name = None
@@ -129,6 +132,9 @@ class CallbackModule(CallbackBase):
 
         task_data = self._task_data[task_uuid]
 
+        if self._fail_on_change == 'true' and status == 'changed':
+            status = 'failed'
+
         if status == 'failed' and 'EXPECTED FAILURE' in task_data.name:
             status = 'ok'
 
@@ -224,7 +230,10 @@ class CallbackModule(CallbackBase):
             self._finish_task('failed', result)
 
     def v2_runner_on_ok(self, result):
-        self._finish_task('ok', result)
+        if result._result.get('changed', False):
+            self._finish_task('changed', result)
+        else:
+            self._finish_task('ok', result)
 
     def v2_runner_on_skipped(self, result):
         self._finish_task('skipped', result)