Tuesday, January 5, 2010

Puzzles and Mysteries

For my second post inspired by Malcolm Gladwell's latest book "What the Dog Saw, and Other Adventures", I want to turn to the chapter "Open Secrets" (originally published in the New Yorker, Jan 2007).

In his analysis of different kinds of intelligence, Gladwell picks up the distinction between puzzles and mysteries originally proposed by Gregory Treverton. (See Curtis Frye's review of his 2003 book Reshaping National Intelligence for an Age of Information. See also Risks and Riddles, published in the Smithsonian Magazine, June 2007. And see my post on Making Intelligence Relevant.)

A puzzle is characterized by having a definite answer, if we can only find it. A puzzle is difficult only because we don't have enough information. For example, Gladwell and Treverton classify the question "Where is Osama Bin Laden?" as a puzzle. If we knew exactly where Bin Laden was, then this would cease to be a puzzle at all. So the purpose of intelligence here is to get relevant pieces of information that help narrow down the field of search.

A mystery is characterized by ambiguity and uncertainty. For example, the question "What is Osama Bin Laden up to?". There is perhaps no shortage of information here, there may even be too much information, the difficulty is in interpreting the information correctly.

Gladwell uses the distinction to discuss the Enron case. He argues against the popular view that Enron management concealed their dealings, and points out that the information was freely available to those that took the trouble to wade through the documents. In contrast to the Watergate case, which was only revealed because an insider (the famous "Deep Throat") leaked information to Woodward and Bernstein, journalists investigating the Enron case simply downloaded the information they needed from the Enron website. Indeed, several years before Enron fell, a bunch of MBA students had carried out a pretty accurate analysis, based merely on the published accounts. In Gladwell's opinion, Enron therefore counts as a mystery rather than a puzzle.

This is consistent with an assertion made by Harold Wilensky in his 1967 book Organizational Intelligence (which Gladwell has cited elsewhere), that a sophisticated reporter working with open sources may achieve more than an agent working with top-secret information. Wilensky highlights the distorting effects a doctrine of secrecy can have on intelligence: one example in Wilensky's book concerns the possible consequences of an American invasion of Cuba, where reporters read the situation more accurately than the CIA experts.

Dishonest people can create puzzles simply by withholding important information. Enron executives went to prison because their conduct was judged dishonest. Gladwell agrees that Enron was reckless and incompetent, but defends the company and its executives against the charge of concealment. The state of Enron's finances was too complicated even for its own executives to understand; we might imagine that some of the executives consciously took advantage of this complexity, but we could equally imagine that this was a situation that Enron blundered into without any deliberate strategy. Hence the mystery,

Characterizing Enron as a mystery also goes some way to explaining why the auditors were useless in detecting the fraud - if it was indeed a fraud. The audit process is designed to spot errors and omissions in the financial accounts, which might indicate dodgy dealings somewhere in the organization. The audit process is not designed to spot excessive complexity or risk, and auditors do not generally practice the kind of ratio analysis that the MBA students used. (Auditing is therefore one of those "best practices" whose flaws are exposed by complexity.)

Another lens through which the Enron accounts could reasonably have been viewed was the taxation lens. The fact that Enron wasn't paying much corporation tax (in several years it paid no income tax at all) might have been seen as an important clue to its lack of real profitability. However, those who wanted to believe in Enron's profitability could easily convince themselves that the low level of tax payments represented clever tax avoidance - in other words, interpreting it as evidence of the smartness of the accountants and/or the stupidity of the tax authorities. (Thus the accountancy lens was used to discredit alternative lenses that might have revealed alternative truths.)

Another way of thinking about the difference between puzzles and mysteries is that puzzles are about people (deliberate conspiracies) while mysteries are about systems. As Gladwell tells the story, Enron wasn't about a handful of bad people misleading everyone else, it was about a system that led everyone astray. Working out a mystery is not a question of collecting more information, but about finding a frame or lens for systematic analysis, to make sense of the information we already have.

Does it make sense to divide intelligence problems into puzzles and mysteries, as Treverton and Gladwell do? I'm not convinced there is a simple either/or, but I'm not sure that's what Treverton and Gladwell are claiming anyway. I  think what is important here is not to identify which problems count as puzzles and which ones count as mysteries, but to acknowledge that at least some problems do count as mysteries in Treverton's terminology, and therefore we need an intelligence capability that helps us to make sense of too much information, and not rely solely on an intelligence capability that merely gathers more information in the hope of resolving something. Examples of this need can be found both in national security and in business.

In terms of organizational intelligence, this means achieving a good balance between two capabilities - the information gathering capability (Perception) and the analysis capability (Sense-Making) - and linking effectively into the remaining capabilities (Decision, Action, Learning). Sometimes merely collecting more information doesn't help solve the problem, especially if we don't have the capacity to interpret the information we already have, or if the new information merely provides an excuse for further procrastination.

(This is similar to the point some of us were discussing recently on Twitter: whether it is always good to produce more ideas, or whether it is sometimes possible to have too many ideas, especially if you don't have the capacity to use them effectively.)

Was Enron intelligent? Enron certainly did some clever and innovative things, but the spectacular failure of Enron suggests that there were some flaws in the thinking. Here are two suggestions. Firstly, a collective failure within Enron to appreciate the scale of its exposure to risk indicates a weakness in sense-making - an insufficiently robust way of seeing beyond the complexity of the accounts and understanding the true financial state of the company. And secondly, a collective refusal to learn from the voices of doubt coming from external critics. When a company is convinced by its own cleverness, this conviction can become a barrier to learning, and therefore a limitation of true intelligence.

Other examples of the puzzle/mystery dichotomy

Related posts: Connecting the Dots, Explaining Enron, Making Intelligence Relevant

No comments:

Post a Comment