The Evidence-Based Revolution
This is how democracy works.
Politician Jones identifies a problem. He proposes Solution X. Many people think Solution X will work. Politician Jones is elected.
Politician Jones implements Solution X. Time passes.
If the problem gets better, Politician Jones credits Solution X and the bold leadership that made it possible; the opponents of Politician Jones scoff and say other, external factors were responsible. If the problem gets worse, Politician Jones blames external factors while his opponents say this conclusively proves Solution X doesn’t work.
And the truth? The truth is we don’t really know if Solution X works or not. Lots of people think they do. But they don’t. And yet, de-spite not having a clue, politicians will expand Solution X, or scrap it, and be sure they’re right to do so.
That is how democracy works. And that is why, if you are a reasonable person who follows politics and government closely, you are de-pressed. So allow me to introduce you to a psychiatrist. His name is Ben Goldacre.
In addition to being a psychiatrist, Goldacre is a doctor, an epidemiologist, a top-tier data wonk, and a self-described “evangelist” for evidence-based thinking. His “Bad Science” column in the Guardian – which dissects and eviscerates science-abusing quacks, journalists, and politicians – became a book of the same name which sold half a million copies around the world. For laypeople, there is no better guide to thinking like a scientist. Best of all, it’s funny as hell.
As a physician, Goldacre is in his element because, today, most people get the idea that medicine should be evidence-based. It’s not good enough to say that Mrs. Smith’s lumbago was dreadful until she took the pill you invented, or to tell a plausible story about why the pill should work, or to wave your credentials around and demand obedience. Intelligent people expect the pill to be properly tested for efficacy and safety – by means of randomized controlled trials (RCT), the gold standard of scientific inquiry – and they won’t swallow it until it has been.
That expectation is the single biggest reason to be optimistic, Golda-cre says. It is a very big and recent change.
“It’s a real mistake to imagine that trials have always been used in medicine,” Goldacre told me at a café in Hong Kong, where we met last week. “That battle was won against an enormous degree of resistance from very eminent, senior doctors. And it was won within living memory.”
It really wasn’t until the 1960s that RCTs started to proliferate in medicine. Even just 25 years ago, doctors often resented attempts to put their treatments to the test.
Which made the Scottish physician Archie Cochrane very un-popular.
“As recently as the ’70s and the ’80s,” Goldacre notes, Cochrane was “sitting down with world experts in cancer treatment and saying how do you know that your operation is better than the operation done by this other professor of cancer surgery down the road?” The answer was always the same. “Of course I know what’s best! I’m a world ex-pert!’ ”
In a legendary showdown in the 1970s, Cochrane questioned whether cardiac-care units, which were new at the time, and used untested interventions, were really beneficial for patients who had survived a heart attack. Cochrane proposed an RCT: Randomly selected patients would be sent home for rest under the supervision of their family physician, while others would go to the cardiac-care units. Outcomes would be compared.
The cardiac-care physicians were furious. “They all said, ‘You’re a madman! Of course what we do is effective!’ ”
But Cochrane pushed ahead and got the trial going. When he had interim results, he called a meeting with the doctors and revealed the data. You were right, Cochrane said. Patients sent home were less likely to survive than those sent to the cardiac-care units. “The room erupted in fury! People called him a murderer for running this trial,” Goldacre laughs. “And then he said, ‘Well, I’m sorry to have to share this with you gentlemen, but actually I played a practical joke on you.’ ” Cochrane revealed the real data, which showed that patients sent home did better than patients in the cardiac-care units. “And he looked up and he said, ‘Am I still a murderous bastard?’ ”
The fact that medicine has changed so dramatically, so recently, should give us hope that other fields can as well, Goldacre argues.
Earlier this year, he and three co-authors wrote a paper for the U.K. Home Office on the role that randomized control trials can play in public policy.
“People often think, ‘How are you going to be able to do a randomized trial on some huge, major, land-mark policy, some huge initiative that a government minister is really closely wedded to?’ ” Goldacre says. “But actually I think the place to start on policy is not on the big glamorous stuff. It’s on the small, trivial stuff.”
In part, that’s because doing so avoids partisan politics. But Golda-cre also believes that’s where major gains are waiting to be made – like the RCT which revealed that a simple change in wording in-creased rates of response to letters reminding people to file their taxes on time. The RCT cost nothing. But the changed wording saved money.
That may sound inconsequential, but even small improvements can make a major difference, Golda-cre says. “In medicine, your chances of dying as a middle-aged man have roughly halved over the past few decades. But that’s not because of any sudden blockbuster. It’s be-cause of a gradual accumulation of tiny, tiny changes.”
Small stuff at the delivery end could also entrench the idea of trials and gradually change the culture in government. Trials would become the norm. They would be expected. It would seem bizarre to introduce a program without proper testing. “Eventually, by getting the method of testing and learning and adapting to be accepted and commonplace and obvious for these trivial questions then after a while you’d find people saying, ‘Well, minister, your fantastic and amazing back-to-work plan that’s going to help people get off (welfare), obviously we’re going to see if that works or not, right? Because that’s what we always do with everything.’ What you need is for it to become so normal that you do it, that it would be just too freaking weird and embarrassing and shameful in the room for them to go, ‘Oh, I’d rather we didn’t.’ ”
But the real breakthrough will come when the public expects policies to be tested just as they expect pills to be tested. What’s needed, Goldacre says, is “a broader expectation that if you don’t test stuff, if you don’t test your ideas, then there’s obviously something wrong with them.”
Maybe he’s a dreamer. But like the best evangelists, it’s hard to listen to Goldacre and not believe.