Measures of training include effects on cognitive processes and effects on the quality of decisions. We also look at subjective measures, such as the effects of training on confidence in decisions and rating of the training by trainees.

Critical thinking processes include the extent to which officer consider critical issues in the scenario, identification of conflicting evidence and conflicting goals, handling cases of conflict (e.g., by re-evaluating assumptions), exploring alternative options, and developing contingency plans in case problems occur. These critical thinking processes are described in the Recognition / Metacognition model.

Measures of decision quality are objective if possible, e.g., with reference to a ground truth vector that is used to generate an experimental scenario. In other cases, measures of decision quality are based on agreement with subject matter experts, as well as agreement among trainees themselves.

Statistically significant effects of critical thinking training have been found with measures of both process and decision quality (singificant effects are shown in red in the table). All changes in both process and accuracy -- whether statisically significant or not -- were always in a positive direction.

Training to think critically did not reduce confidence, and in fact tended to slightly increase it.

Officers' subjective evaluations of the training were also consistently positive. There were many positive comments about specific aspects of the training, e.g., the devil's advocate method, as well as about the general sharpening of thinking skills. By far the most common negative comment was "I already do this."