Diamond Head

"Do you mean to say they could be rounding Diamond Head and you wouldn't know it?" The man who asked this was Husband E. Kimmel, commander in chief of the US Pacific fleet, and he addressed the question to Edwin Layton, his intelligence officer. "They" were the Japanese aircraft carriers, and Diamond Head is a famous mountain peak on the Hawaiian island of Oahu, just 20 kilometres from Pearl Harbour. The date was December 2, 1941. Kaga, Akagi, Hiryu, Soryu, Shokaku and Zuikaku were not off Diamond Head, but they had left port seven days before, to launch an attack that would start a war - and incidentally, bring an inglorious end to Kimmel's career.

When that attack came, Kimmel had done nothing to prevent it or blunt its impact, although his question showed that he was not entire unaware of the danger. Intelligence failure is of all times, and when it happens it is common to say, as is done now after the attack in Detroit, that it was a failure to connect the dots. That is a convenient explanation, but it doesn't do any justice to the problem of analysing a diverse collection of bits of information into useful intelligence. In reality, the paper tends to be full of dots, some of them true and some of them false, and the problem is one of selecting the relevant dots and connecting them in the right sequence. Usually the problem cannot be solved by studying all possible combinations, because the number of these is simply too vast, and a method must be devised to create the potentially most useful ones. Even then, every picture that emerges has to be assessed for the probability that it is real. Kimmel, as his question illustrates, did not fail to connect the dots in the correct threatening picture, but he dismissed it as too improbable.

In 1976 president Ford agreed to have two teams conduct a parallel analysis of all intelligence the USA had about the military capabilities of the Soviet Union. Team A consisted of the professional experts of the CIA. Team B was deliberately composed from people outside the intelligence community: Among the names associated with this effort, we encounter Dick Cheney, Donald Rumsfeld, Paul Wolfowitz, and Richard Perle. Both teams were given full access to the same information. Needless to say, a completely different picture was produced by the two teams. It is perhaps unsurprising, given the involvement of the people mentioned above in the 2003 intelligence fiasco about Iraqi WMD, that Team B's specific claims about Soviet weapons systems later turned out to be "all wrong. All of them." Yet these people had the same dots to connect. The  difference was generated by their assessment of which patterns were the most valuable ones. Team B started from the assumption that worst-case scenarios deserved more credence, because Soviet intentions were hostile. That turned out to be a wretchedly poor criterion, both in 1976 and in 2003.

Paranoia is a poor guide in such matters. Occam's Razor may be a better one. R.V Jones, who provided Churchill with information about German weapon systems during WWII, advised his successors in 1947 that "You will find yourself confronted with many frightening bogies conjured up by the agile imaginations of men often at higher levels than yourself. You will be unable to lay all these bogies at once, because to prove a negative case is one of the most difficult of intelligence exercises. But you must find the simplest, commonsense hypothesis and stick to it until a fresh fact proves you wrong, however eminent an authority is attached to another view." Occam's Razor has the disadvantage that it does not weigh possible scenarios for risk. Clearly an interpretation that is only moderately likely but carries a very high risk, deserves to be investigated. But it is all too easy to dream up scenarios that combine astronomically high risk with vanishingly low probability --Hollywood script writers do so all the time-- and Occam may at least help us to eliminate these.

The big question today is how the analysis of intelligence can be improved. In the USA, the post 9/11 analysis suggested a bureaucratic solution: The office of the DNI, or Director of National Intelligence. More than anything else this marked a phase in the long struggle between the various civilian and military intelligence agencies, which were united in disunity by making them all report to the DNI. Characteristically, if the Director is a civilian then the Principal Deputy Director is required to be military officer, or the other way around. That this would not actually improve the situation much should have been obvious, and it probably made things worse by adding even more bureaucratic layers between analysts and policy makers. Such layers don't just pass information, but they rewrite, reinterpret, edit, summarize, and filter. When an intelligence analysis that was prepared by a professional who had the facts at hand, passes through the hands of a number of amateurs who don't, information is inevitably lost.

There is tendency to regard intelligence analysis as an arcane practice, as complex and secretive as the ritual of some eccentric sect. This bans professionals to quiet back rooms, where members of arcane sects belong, while the levers of interpretation and power remain comfortably in the hands of bureaucrats and politicians. This situation is harmful and unnecessary. To get accurate and relevant intelligence in a timely manner, it needs to be handled by professionals. There is nothing particularly novel about this: Since the 19th century, it has been military practice has been that every operational commander had an intelligence officer on his staff.