The Shruti model and implementation have been extended in a number of ways, including: improved abductive reasoning; multiple-consequent rules; soft / evidential rules, facts and evidence combination calculus; priming and recency effects; supervised learning via backpropagtion in the structured network; and, most importantly, the model has been augmented to integrate the propagation of "belief" with the propagation of "utility". This integration will allow the model's sense of utility to direct its search for answers and explanations and focus its activation along paths that promise to have a high utility. We have continued to develop the connections between the recognitional and metacognitive systems.

This work includes an experiment in which we tested the efficacy of a metacognitive "intervention" on the construction of a situation estimate and on learning in the reflexive system. The machine learning experiment utilized one of the core scenarios identified in the tactical decision making domain of the hybrid computational architectures program (Korea). In this scenario, some kinematic cues suggest hostile intent (an aircraft flying at low altitude toward own ship). Others, however, do not (speed is only moderate), and there is also the possibility that the aircraft is being deployed with the intent of defending another platform against U.S. ships. We developed an event tree to reflect branching possibilities of key events in this scenario. Among these possibilities were hostility between the US and the other country, the appropriateness of the approaching contact for use as an attack or rescue platform, intent (to attack U.S. ships, or to protect the other platform from the same country), and kinematic features of the track (bearing, altitude, and speed). Probabilities were assigned to the branches of this tree in order to create a set of scenarios representative of causal / evidential relationships in the real-world domain underlying the scenario. A randomly selected half of these scenarios were used for training, and half for testing. Backpropagation was used to adapt the weights of the reflexive system in the training scenarios. In the reflexive-only condition, each scenario was trained for 50 cycles of relaxation. In the metacognitive training condition, a hint to "think about" (i.e., query) non-hostile intent was inserted midway through these relaxation cycles.

In both learning conditions, rules for concluding hostile intent from purely kinematic cues (e.g., heading toward own ship at low altitude) were reduced in strength. However, changes in the knowledge base were more pervasive in the metacognitive learning condition, and applied to inferences of hostile intent from contextual as well as kinematic cues. For example, in the metacognitive condition, the reflexive system learned that when tensions between two countries are high, the intent of a military platform may be to protect other platforms rather than attack own ship. A series of four behavioral studies has examined the effect of training metacognitive skills on tactical decision making by 201 active-duty Naval officers in scenarios similar to the core scenarios. The training emphasized four aspects of metacognition, which were presented as the S.T.E.P. technique for tactical decision making:

* Story-building: When faced with a situation in which events are difficult to interpret, stakes are high, and time allows for deliberation, formulate a story that motivates and explains past and present events.

* Testing the story: Ferret out evidence that conflicts with this story. Try to generate an explanation for this evidence that preserves core aspects of the story, such as your assessment of the intent of a suspect track.

* Evaluating the story: Judge the plausibility of the story you have generated, bearing in mind assumptions you have made to handle conflicting and missing evidence. If the story is weak, build a new story, test it, and evaluate it.

* Planning against weak assumptions: Formulate contingency plans to protect against the failure of key assumptions.

Officers in these experiments executed high-fidelity AAW scenarios on computer-based simulators, and, in written tests, assessed the intent of experimenter-selected tracks from each scenario, critiqued assessments, and defended them. From these data, we assessed the effects of training on tactical decisions by comparing the accuracy of officers' assessments with those of an SME. We evaluated the effects of training on metacognitive processes with counts of the arguments officers made concerning assessments.

In each of the four experiments, every reliable effect of training on metacognitive processes was positive and large (ranging from 23% to 94%). Furthermore, the mean score on each process measures rose with training in every experiment, and though some of these increases were not statistically reliable, the pattern over all studies and all measures was consistently positive.
 

Experiment	#1	#2	#3	#4
Metacognitive processes
Story-building: Defending preferred assessments	+	+23%**	+26%(p<.10)	+
Story-building: Defending non-preferred assessments	+	+23%**	+40%*	+
Testing stories: Identifying conflict	+52%*	+58%**	+	+94%*
Testing stories: Explaining conflict	+	+27%**	+23%*	+
Evaluating stories: Generating alternative assessments	+	+41%**	+	+48%*
Other
Confidence in preferred assessment	+	+18%*	+	+

Table #1: All reliable effects on measures of metacognition were positive, and mean scores on all measures rose with training in all experiments. ** = p < .01; * = p < .05; + = positive but statistically unreliable improvement with training.

Metacognitive skills training also had positive effects on the accuracy of tactical decisions. In every case in which training had a statistically reliable effect on decision accuracy, that effect was large and positive. The smallest effect on accuracy was an improvement of 35%; in the largest case, training produced accuracy rates of 83% vs. 16% among controls. In addition, officers' confidence in their assessments rose at the mean in every study. (The effect on confidence was reliable in only one experiment, however).

In sum, training metacognitive skills appears to improve decisions and the processes by which officers make their decisions, and also to improve the process by which the hybrid architecture adapts its beliefs.