A widely accepted research paradigm in psychology
(e.g., Kahneman, Slovic, and Tversky) has taken Bayesian decision
theory as the standard by which reasoning is to be judged, and
has identified pervasive patterns of error, called biases, in
laboratory performance. According to these researchers, unaided
decision processes employ rules of thumb (or "heuristics") that
under many (but not all) conditions lead to "severe and systematic
errors"
Evaluating Decisions and Decision Making
CTI has investigated decision-making errors in a variety
of real-world contexts, such as submarine command decision
making, Air Force in-flight route replanning, and Army battlefield
command staff.planning. This work has been accompanied by
a thorough review and analysis of psychological research on
decision making, and by normative research on uncertainty,
inference, and belief revision. The result is a view of
biases based more solidly on both data and theory.
The point of view we take is naturalistic,
that is, rooted in study of the processes used by real-world
decision makers and their real-world outcomes. From this point
of view, formal consistency (e.g., with Bayesian probability
theory) is only one among the factors that should be considered
in evaluating a decision making process. For example:
Effort versus accuracy in strategy choice: Do decision
makers select strategies that appropriately balance accuracy
against time and effort?
Cost and benefits of thinking more: Do decision makers
appropriately determine how long to keep options open versus
committing to action, as a function of the dynamic character
of the environment, the costs of delay, and the opportunity
to obtain new information? (This is the control process
we call the Quick Test in the Recognition
/ Metacognition model.)
Strategy monitoring: Do decision makers monitor the use
of a strategy to guard against its potential shortcomings
in specific contexts? (This is the critiquing
process identified in the Recognition
/ Metacognition model.)
Incremental improvement: Do decision makers take opportunties
to incrementally improve their understanding of relevant
beliefs, preferences, and options as a problem evolves?
(This is the correcting process in the Recognition
/ Metacognition model.)
Do decision makers approriately evaluate their own knowledge
or expertise in a domain?
Do decision makers persist in holding to a belief or decision
after a significant accumulation of negative reasons, and
in the presence of a better alternative? (This is part of
the process of building and evaluating mental
models.)
Do decision makers abandon a belief or decision too readily
in the face of negative reasons that can be explained relatively
easily, or in the absence of a better alternative?
The Confirmation Bias
Points 6 and 7 bear on the confirmation bias,
a tendency to interpret negative evidence as favoring a preferred
hypothesis, or mental model.
Contrary to much of the literature, this tendency to explain
away inconsistent evidence is NOT necessarily an error. In
many situations that we have observed, no familiar pattern
fits all the evidence; there are data that conflict with every
plausible hypothesis. The traditional view of the confirmation
bias either paralyzes decision makers or forces them to settle
for an unrealizable statistical average of the possibilities.
Our research suggests that experienced decision makers try
to make sense of conflicting data by constructing coherent
stories, i.e., by revising and improving their
mental models through a process
of critiquing and correcting.
They then step back and evaluate the plausibility of the story.
If it requires too many unreliable assumptions, they try to
construct a different story, step back to evaluate it, and
so on. This iterative process is part of what we set out to
capture in the Recognition / Metacognition
model of critical thinking about mental
models. It is also reflected in our normative work on
assumption-based reasoning and connectionist
models that integrate relfexive
and reflective reasoning.
Finally, this more dynamic and adaptive view of decision
biases underlies the Personalized
and Prescriptive approach to decision aiding.
