Tuesday, November 13, 2007

"Expectancy bias" in the USS Vincennes shootdown

I've just started dipping into what looks like a great book: "Sources of Power - How people make decisions" (Gary Klein, MIT Press, 1999). It has a fascinating chapter on the shooting down of an Iranian commercial jetliner in 1988 by the USS Vincennes.

Klein says that some decision making experts see this terrible incident as an example of "expectancy bias" (pp 84ff). Specifically, the crew in the Vincennes' Combat Information Centre reported seeing the unidentified aircraft descending towards the ship - in a pattern they saw as being aggressive. In fact - the data shows that the aircraft was always ascending through all stages of its short flight (it was shot down only 7 minutes after take-off).

What explains this mis-reading of the flight of the airliner? The decision making experts mentioned by Klein think the Vincennes crew were already convinced that this was an enemy jet. Why? Just after take-off there had been a (later shown to be false) IFF reading that the airliner was an enemy jet - and not a civilian plane (the full account of the whole incident in this chapter is worth reading). Because of this belief, the experts claim, the crew then started to see the data they expected to see: the data that supported the scenario they had already decided was occurring. A classic case of "expectancy bias".

Klein is not so sure. He asks the question - what would have happened if the Vincennes had not shot down the unidentified aircraft and it turned out to be an enemy jet that then successfully attacked them? Well - over the preceding month there had been 150 challenges issued by US ships in the Gulf to presumed enemy aircraft. In over 80% of those cases the aircraft involved turned out to be Iranian military (and only 1.3% turned out to be commercial airliners). In other words, if the Vincennes had not acted, and it did turn out to be the wrong call (that is - the aircraft attacked them) - then this decision could also be explained by "expectancy bias". It's just that in this case the bias would have been different - a bias against the available data that these unidentified jets turn out to be Iranian 80% of the time.

This is a great insight from Klein. "Expectancy bias" explains why the Vincennes shot down the airliner. But "expectancy bias" would also have explained the opposite scenario - the failure to shoot down an enemy jet that then successfully attacked. In Klein's words expectancy bias "explains too much" (p85). A theory that explains too much is no theory at all.

So why did the Vincennes make such a tragically wrong decision? Well, it seems that after a terrible chain of events, it was at the end a simple failure of information design. On the Vincennes' computers, the altitude data for nearby aircraft was contained in a small display off to the side. And critically, it did not display a trend: crew-members had to remember the previous altitudes of the jet so as to see if it was going up or coming down. In a pressured situation - over a short period of time - this turned out to be too hard.

2 comments:

Tom said...

As an ex RN CPO I would like to ask why none of the photographs or film of the crew of USS Vincennes were wearing anti-flash protection? They had plenty of time to don this equipment yet they were all dressed as though they were on a pleasure cruise. Either they disregarded what must be standing orders, or they KNEW that they were not under attack.

Tom said...

As an ex RN CPO I would like to ask why none of the photographs or film of the crew of USS Vincennes were wearing anti-flash protection? They had plenty of time to don this equipment yet they were all dressed as though they were on a pleasure cruise. Either they disregarded what must be standing orders, or they KNEW that they were not under attack.