Dan Gardner is the New York Times best-selling author of Risk, Future Babble, Superforecasting (co-authored with Philip E. Tetlock), and How Big Things Get Done (co-authored with Bent Flyvbjerg). His books have been published in 26 countries and 20 languages. Prior to becoming an author, Gardner was an award-winning investigative journalist. More >

What we can learn from Paul the octopus

Nobody takes Paul the Octopus seriously. We're too clever for that. As most of the planet knows by now, Paul the Octopus, the star attraction at an aquarium in Oberhausen, Germany, was asked to "predict" the outcome of Germany's matches at the World Cup by choosing between two boxes placed in his tank. One box bore the flag of Germany; the other, the flag of Germany's opponent. Inside each box was a tasty mussel. Paul's first stop for a snack determined the winner. Paul went eight-for-eight, including correctly picking Spain to beat the Netherlands in the final. He even nailed Serbia's upset win over Germany in the opening round. But still, we don't take Paul the Octopus seriously. It's his octopusness. Octopi may be wonderful creatures but they are not known for their intelligence, good judgment, and acute understanding of elite sporting events. Even the sort of doe-eyed innocent who reads his horoscope, swears by homeopathy, and thinks coincidences have cosmic significance will find it hard to see Paul the Octopus's perfect record as the product of anything but luck. But what if Paul the Octopus had been Paul the Pundit? And what if Paul the Pundit had provided plausible rationales for his picks -- and gone eight-for-eight? You can be sure the reaction would have been a little different. Why, that would be a demonstration of such acuity and skill as to make the gods smile and gamblers weep. All hail Paul the Pundit! Interviews and celebrity would follow. So would book deals and lucrative speaking engagements. Only the occasional oddball would dare to suggest that maybe Paul's perfect record was more a product of randomness than wisdom. After all, the odds of correctly calling a flipped coin eight times in a row are one in 256. And literally millions of people -- and thousands of pundits -- made predictions about the World Cup. So it's only to be expected that some lucky people will finish the tournament with perfect predictive records. A dart-throwing chimpanzee could do it, or a mussel-eating octopus. Of course, the oddballs would be ignored. Paul the Pundit would celebrated as a wise man, a guru, an oracle. And he would cash in. I know that sounds a little cynical. But a process similar to this happens quite routinely. And in fields far more consequential than sports. Consider Bill Miller, a legendary fund manager who beat the S&P 500 15 years in a row. It was an amazing record. No one was better. People who follow the financial world know all about randomness, but they were sure this couldn't be luck. The odds of chance alone producing such a result are infinitesimal, they said. It must be skill. But as Leonard Mlodinow explained in The Drunkard's Walk, Miller's fans made a remarkably basic error: They looked at Miller alone and said, "what are the odds of this happening by luck to him?" And since those odds are indeed tiny, they concluded it wasn't luck that accounted for Miller's streak. But Miller wasn't alone. There are lots of fund managers. What people should have asked is "what are the odds of this happening by luck alone to at least one of the many fund managers?" When Mlodinow made a very conservative calculation, he found that over a 40-year period the odds of some lucky manager scoring a 15-year winning streak are almost three out of four. What's most remarkable about this story is that sophisticated people made such a dumb mistake. It's as if we got all excited after some guy in Rimouski won the lottery because, hey, the chances of that guy picking the right numbers by luck alone are so tiny they are effectively zero! So it can't be luck, right? He must be psychic! We make these sorts of mistakes all the time. Vast numbers of predictions are made. Most fail. A few succeed. Occasionally, someone gets a streak of predictions right, or nails a particularly big event. We celebrate his insight. We read his books and articles and pay top dollar to sit in an audience and soak up his wisdom. The wealthy and the powerful invite him to their conclaves to share his vision. He is an oracle. Until his luck runs out, that is. When his predictions start flopping, he's a chump. And soon, he's forgotten. Now, it would be forgivable if we made this mistake once and learned from it. But we don't. When an oracle is revealed to be nothing more than a guy who got lucky, it's only that particular oracle, not the idea of oracles, that is discredited. So we go looking for somebody else "who called it." And we usually find one because -- like lotteries, and for the same reason -- there's usually a winner. This long and dismal record inspired J. Scott Armstrong, an expert on forecasting at the Wharton School of Business, to create his Seer-Sucker Theory: "No matter how much evidence exists that seers do not exist, suckers will pay for the existence of seers." There are many reasons for our continual failure to dispense with the idea of oracles, as I discuss in Future Babble, my next book, set for release in October. (For those keeping track, that's the 11th shameless plug I've made in this space.) Most stem from our profound psychological aversion to uncertainty -- we believe because we want to believe -- but some are more basic. One is our unfortunate habit of paying attention to hits while ignoring misses. This is why horoscopes seem to provide insight: We zero in on the information that seems relevant to our lives, and we remember it, while we pay little attention to the rest and immediately forget it. This biased processing of information ensures even when the misses vastly outnumber the hits, the hits will seem to outnumber the misses -- a misperception that has been profitable for psychics and soothsayers since the dawn of time. This applies to clever people who would never dream of taking horoscopes seriously. Consider Nouriel Roubini, the economist who shot to global gurudom after he correctly predicted the meltdown of 2008. Every time he is interviewed, every time he is introduced to an audience, Roubini's famous call is mentioned. What is never mentioned is that Roubini also called for a recession in the United States in 2004, 2005, 2006, and 2007. Or that after the crash of 2008 he said oil would stay below $40 a barrel throughout 2009 (it doubled in price). Or that he said stocks were going nowhere but down (they soared). I'm not saying Roubini simply got lucky in 2008. He's a smart guy with lots of genuine insight. But he's no oracle. There are many other forecasters whose inflated reputations depend on people touting the hits and ignoring the misses. Former CIBC chief economist Jeff Rubin is one. Another is Paul the Octopus. Yes, Paul the Octopus. What few have noted about that glorious cephalopod is that World Cup 2010 was not his first foray into prediction. Euro 2008 was: He got a mediocre four right out of six calls. There are no oracles, not even Paul the Octopus.