How Doctors Think

@nybooks I've been looking at the work of Jerome Groopman, author of How Doctors Think, as well as a number of recent articles in the New York Review of Books.

• February 11, 2010: Health Care: Who Knows 'Best'?
• December 17, 2009: Dilemmas for Doctors
• November 5, 2009: Diagnosis: What Doctors Are Missing 

---

Errors of judgement

Groopman discusses the seminal work of psychologists Amos Tversky and Daniel Kahneman, who challenged the prevailing notion that the economic decisions we make are rational. (This field of study is  known as behavioural economics.) We are, they wrote, prone to incorrectly weigh initial numbers, draw conclusions from single cases rather than a wide range of data, and integrate irrelevant information into our analysis. Such biases can lead us astray.

It seems doctors are not immune to such cognitive errors.

Some 10 to 15 percent of all patients either suffer from a delay in making the correct diagnosis or die before the correct diagnosis is made. Misdiagnosis, it turns out, is rarely related to the doctor being misled by technical errors, like a laboratory worker mixing up a blood sample and reporting a result on the wrong patient; rather, the failure to diagnose reflects the unsuspected errors made while trying to understand a patient's condition.

Groopman quotes an associate, a senior cardiologist, who made fatal errors by relying too heavily on logic.

"Impeccable logic doesn't always suffice. My mistake was that I reasoned from first principles when there was no prior experience. I turned out to be wrong because there are variables that you can't factor in until you actually do it. And you make the wrong recommendation, and the patient doesn't survive. I didn't leave enough room for what seems [sic] like minor effects."

---

Best Practice

As a way of improving diagnosis, there is growing reliance on "clinical guidelines", the algorithms crafted by expert committees that are intended to implement uniform "best practices".

Behavioural economics provides a justification for the proliferation and dissemination of "best practices" to healthcare professionals. If doctors are prone to systematic errors, then best practices will protect them and their patients from the consequences of these errors. Medical best practices are encouraged from President Obama downwards.

But there are some big problems with this approach, as Groopman explains.

Over the past decade, federal "choice architects"—i.e., doctors and other experts acting for the government and making use of research on comparative effectiveness—have repeatedly identified "best practices," only to have them shown to be ineffective or even deleterious.

What may account for the repeated failures of expert panels to identify and validate "best practices"? In large part, the panels made a conceptual error. They did not distinguish between medical practices that can be standardized and not significantly altered by the condition of the individual patient, and those that must be adapted to a particular person. For instance, inserting an intravenous catheter into a blood vessel involves essentially the same set of procedures for everyone in order to assure that the catheter does not cause infection. Here is an example of how studies of comparative effectiveness can readily prove the value of an approach by which "one size fits all." Moreover, there is no violation of autonomy in adopting "aggressive" measures of this kind to assure patient safety.

But once we depart from such mechanical procedures and impose a single "best practice" on a complex malady, our treatment is too often inadequate. Ironically, the failure of experts to recognize when they overreach can be explained by insights from behavioral economics.

---

Next Practice Tools

How Doctors Think has an epilogue with advice for patients. Groopman gives the following tools that patients can use to help reduce or rectify cognitive errors:

• Ask What else could it be?, combating satisfaction of search bias and leading the doctor to consider a broader range of possibilities.
• Ask Is there anything that doesn't fit?, combatting confirmation bias and again leading the doctor to think broadly.
• Ask Is it possible I have more than one problem?, because multiple simultaneous disorders do exist and frequently cause confusing symptoms.
• Tell what you are most worried about, opening discussion and leading either to reassurance (if the worry is unlikely) or careful analysis (if the worry is plausible).
• Retell the story from the beginning. Details that were omitted in the initial telling may be recalled, or different wording or the different context may make clues more salient. (This is most appropriate when the condition has not responded to treatment or there is other reason to believe that a misdiagnosis is possible.)

[source: Wikipedia]

Plenty of good advice there, not just for healthcare professionals but for anyone dealing with complex system problems.

Early Warning Signals

As @bmichelson reports, rockstar Dave Lee Roth used to demand a bowl of M&Ms in his dressing room, with all the brown ones removed. Why? Because he wanted to test the venue's attention to detail. If they couldn't even get the sweets right, how could he trust them to correctly install the complex wiring and lighting required for the gig? If the test failed, what that triggered wasn't a childish tantrum but righteous anger and a thorough test of everything else before the band would go on the stage. [Business Advice from Van Halen, Fast Company March 2010 via Elemental Links] As @bmichelson points out, this can be regarded as a form of instrumentation.

I have seen a similar trick described somewhere else, possibly in one of Mark McCormack's books. When discussing a major event with a large hotel, casually ask for orange juice for all participants, freshly squeezed on the premises that morning. If hotel management appears to regard this requirement as trivial, then this is a warning that it may not take the other requirements seriously either.



Obviously these tricks only work when they are secret. As soon as people realise that these tricks are being used as tests or predictors of performance or quality, then they will alter (distort) their behaviour accordingly. (I'm imagining a scene from a Spinal Tap remake in which a rival rock band sends its roadies to tip off the venue and disrupt proceedings, so that the M&Ms are perfect, but the wiring electrocutes the guitarist and wreaks the concert.)

So if you are going to use these kinds of trick as a shortcut to detect what's going on, you need four things.

1. to design a test that is a good indicator of incompetence, laziness or inattention
2. to behave congruently as if the test really mattered
3. to act appropriately when the test fails (e.g. test the wiring, look for an alternative venue)
4. to watch out for signs that the test has been rumbled

And if you suspect that these kinds of tricks are being used against you, to test your competence and attention to detail, then you are probably already paying enough attention to detail to be able to do an all-round competent job. Maybe next time they'll trust you a bit more, okay?

From "Organizational Intelligence" to "Ability to Execute"

There is a common organization structure found in many large professional firms, including global consultancies and the big industry analyst firms.

For any important topic, we may suppose that the firm employs some of the most knowledgeable people in the world, who are available to give presentations to major clients as well as keynote speeches at international conferences. These high-profile experts attract lots of business to the firm, but of course they don't have time to get heavily involved in the day-to-day work, which is mostly executed by their less famous (and less expensive) colleagues.

It is interesting to consider how this structure affects the overall capabilities of such a firm for handling different kinds of client problem. Where a client problem fits neatly within a single knowledge domain, then we might expect it to be handled reasonably efficiently and effectively, according to the "best practice" principles defined by the centre of excellence for that domain, and with some benefits from the economies of scale and scope. But for large and complex problems spanning multiple knowledge domains, the lack of coordination between the knowledge silos seriously impairs the ability to execute, and the economies of scale are outweighed by the diseconomies of scale.

Ironically, it is precisely these large complex problems where the large global firms claim superiority over their smaller more agile competitors. But this claim is not compatible with an organizational structure based on "best practice". The challenge here is not just building more effective knowledge management platforms (with fancy Enterprise 2.0 software) but architecting the organization to achieve real organizational intelligence.