Why you can't believe predictions

(Money Magazine) -- Stephen Roach has changed his mind. In early May, the widely followed chief economist at Morgan Stanley declared that the economy might not be going to hell in a handbasket after all.

A famous pessimist, Roach had warned for years that the combination of huge trade and budget deficits, low personal savings and overpriced housing could set off a crisis. Then, suddenly, he was saying it might turn out okay. To market watchers, this was Big News.

To anyone who took Roach's call as sufficient reason to load up on stocks, however, it was Big Trouble. Not long after he made his comments, the U.S. and world stock markets took a tumble, in part because Wall Street suddenly got worried about the very risks Roach had been warning about.

Some wags even called Roach's about-face a classic sign of a peak. "When the last bear becomes bullish," quipped a Financial Times pundit, "there is good reason to be worried."

Roach retorted that he was talking about the economy, not stocks, and that he never said the risks had actually gone away.

This isn't meant to pick on Roach (obviously, it's way too early to tell whether he was right). But it's yet one more example of how hard it can be to use an expert's predictions to make money.

Former Fed chairman Alan Greenspan was surely onto something when he wondered aloud about "irrational exuberance" in 1996, but bears who took him too seriously absented themselves from a stock rally that lasted three more years.

Pimco's Bill Gross, one of the world's biggest (and best) bond managers, has been just plain wrong for most of the past year, auguring that the Fed was about to stop raising interest rates, even as it kept raising them.

Managers of large-cap mutual funds, for their part, are so eminently fallible when it comes to predicting the future of the stocks they buy that, over the long run, most of them can't beat a simple S&P 500 index fund.

As for a rather less well paid expert (make that "expert"), the author of this article wrote last year that "we may have seen the worst of the spikes" in oil prices. Remind us next time to ask Ahmadinejad first.

We'll stop there. This isn't going to be another one of those stories that says experts are all gas bags and urges you to ignore them entirely.

"That's too extreme," says Philip Tetlock, a psychologist and professor at the University of California-Berkeley's business school and one of the world's foremost experts on, well, experts. "Certainly experts know less than they claim to know. But that's not the same as knowing nothing."

Tetlock has found that experts in general, just like Wall Streeters, have a pretty dismal record at foretelling. But he has also discovered that some are more accurate than others, not because of what they think but how they think. From the example of the best thinkers, you can actually learn how to think about your own future more clearly.

But you first have to give up the idea that expert predictions are anything more than speculative guesses. And that could be tough because humans - especially we investing humans - crave fortune-telling.

The world is full of surprises, and experts are often just as surprised as the rest of us. Neither of these facts is unique to the people who help you invest your money.

Starting in the 1980s, Tetlock surveyed professional know-it-alls, including academics, think tankers and journalists, and asked them to make predictions about future events around the world.

The results, published last year in his book Expert Political Judgment, are pretty humbling. The experts he surveyed did no better with predictions in their field of study than "dilettantes," experts from other fields who were just drawing on their general knowledge. Some, in fact, did significantly worse.

"The moderately attentive reader of good newspapers can do as well as someone who devotes many years of study to predicting whether, say, Chinese growth rates will continue or Japan's Nikkei index is going up," says Tetlock.

Ouch.

One reason the experts get away with dubious results is that we let them. Our craving for predictions seems to be more deeply entrenched than any innate sense of skepticism.

But the prediction bug is also bound up with a built-in urge to control our fate. (Or at least to act as if we do, as Vegas craps players do when they blow on their dice.) Without a sense of control, we'd be paralyzed.

"You are forced to make forecasts all the time," says Randell Moore, the editor of Blue Chip Economic Indicators, a monthly survey of economists' predictions. You make a risky prediction every time you take a job or start a business.

Hey, you have to start somewhere. During World War II, future Nobel-prizewinning economist Kenneth Arrow worked as an Army weather forecaster. At one point his colleagues alerted the higher-ups that long-term forecasts didn't work. "The Commanding General is well aware that the forecasts are no good," came the reply. "However, he needs them for planning purposes."

Some things in life are, of course, easy to predict: The sun will rise in the east tomorrow, Oprah will be renewed for another season and so on. But experts who peer into the financial future face three daunting challenges.

CHANGE COMES FROM NOWHERE Nassim Nicholas Taleb, author of the cult financial book Fooled by Randomness, points out that the economy is especially susceptible to "black swan" events. This is a riff on a key principle from philosophy, which states that no matter how many white swans you see, you can never be certain that all swans are white.

Taleb believes that black swans - events that defy everything we have come to expect based on past experience - actually waddle along fairly often in markets. "These events are neither repeatable nor modelable, and they dominate the outcome," says Taleb. The technology bubble was in many ways a black swan event. A lucky few made their fortunes from this one-time surprise.

(Many others, of course, blew up their retirement portfolios getting in - and out - too late.)

It would be bad enough if all you had to worry about were big shocks like that. But sometimes it's the little stuff that changes everything. Tetlock points to a classic idea from probability theory. Imagine a bucket containing a red ball and a white ball. You draw one ball from the bucket at random and then put it back in along with another ball of the same color.

You repeat until the bucket is full. If in the early stage you happen to draw a red ball more often, it can be hard to change direction - you're going to end up with a lot of red balls in the bucket. But it's impossible to predict in advance how this quirky game will go.

Economist W. Brian Arthur has noted that any number of college towns might have seemed, in the 1950s, like potential high-tech meccas. But William Hewlett, David Packard and a few other whizzes happened to live near Stanford University. They attracted others, and the Santa Clara Valley became Silicon Valley.

COMPETITION CHANGES THE GAME It's tough enough to predict whether, say, a software company will succeed. But when you invest in that company, you have a whole new problem. If other people share your outlook, they'll buy the stock too, which pushes up the price and makes it harder for you to make money.

In other words, you have to not only be right, but be right when the market is wrong.

"At any point in time, the markets on your computer screen reflect the thinking of countless analysts and forecasters who are making bets on the future," says Roach. The financial forecaster's job, he says, is to find a reason today's consensus will be proved wrong.

The trouble is that this majority is pretty well informed - just about everyone on Wall Street (and in Tokyo and London and Frankfurt) has access to the same data.

WE'RE GOOD AT FOOLING OURSELVES Since the 1970s, a branch of economics called behavioral finance has demonstrated the many ways our perceptions often steer us wrong in judgments about numbers in general and money in particular.

For example, our brains love to detect patterns, even when they're not there. The classic case is the idea that athletes' odds of hitting a ball or sinking a basket go up when they've done it a few times in a row. (See the box on the next page.)

This "hot hand" illusion translates easily from sports to, say, the Janus mutual funds, which after a few strong years in the late 1990s were taking in almost one out of every three new dollars invested in funds. Then the hot hands at Janus cooled, and some of its key funds racked up double-digit negative returns three years in a row.

Sometimes we unconsciously select some facts and ignore others to construct a better tale. Tetlock points out that this can be a particular problem for experts, who, after all, have more information than the rest of us and so can weave great yarns.

Back in 1999, analysts who understood Dell Computer raved that the PC maker had perfected a superefficient production system that let it get paid by its customers before it had to pay its suppliers. Once you understood this, you'd see that Dell was a revolutionary business.

What the storytellers ignored was a fact that any investor should have spotted instantly: A share of the stock cost more than 100 times earnings. It's hard to make money on any stock that expensive, Dell included.

A fragment of Greek poetry reads, "The fox knows many things, but the hedgehog knows one big thing." In 1953 the intellectual historian Isaiah Berlin published a famous essay that divided the world's great writers into hedgehogs and foxes. Tetlock, in turn, has applied that idea to experts.

Foxes, he says, don't struggle to fit everything into an overarching theory of how the world works. They take information from many sources, including those they disagree with, and incorporate it into their thinking. Foxes use language like "but," "however" and "on the other hand." They can be dull, and they don't like extreme predictions, so they probably won't call the next big market turn. But Tetlock has found that foxlike thinkers are, overall, more accurate.

The hedgehog is an "in addition to" and "what's more" kind of guy. He fits everything into one Big Idea, and that's all he'll want to talk about. "When there are home runs to be hit, hedgehogs are more likely to hit them," says Tetlock. Tech guru George Gilder was at first very right when he advocated "telecosm" stocks like JDS Uniphase and Global Crossing; then he was spectacularly wrong. Ironically, it's often hedgehogs who dominate the debate because they're a lot sexier than the plodding foxes.

The answer isn't to simply ignore hedgehogs and follow a fox, though. Tetlock's data show that foxes are pretty darn fallible too. But the fox's open approach isn't a bad model of how we can all start to improve our judgment. Indeed, there's growing evidence that combining a range of forecasts, even if they're extreme and hedgehog-like, is usually far better than listening to just one. Researchers at the Federal Reserve Bank of Atlanta, for example, tracked the accuracy of economists' predictions since 1986 and found that no single expert was as reliable as the average of them all.

Obviously, you're not really going to calculate the average forecast on everything, but you can make sure to incorporate many views in your decisions. For starters, just make fewer decisions - a well-diversified portfolio can get through a variety of possible futures. Then, if you do decide to make a move, don't act until you find a view from someone who disagrees.

"One test of a good forecaster is that they can describe the opposing view," says Ethan Harris, chief U.S. economist at Lehman Brothers. The same could be said of a good investor. Finally, write down the thinking behind any big decisions you make so that you learn from your errors.

Tetlock notes that we are very, very good at forgetting our mistakes.

What experts may be best for, in the end, is helping you diversify your thinking. While you might not agree with an expert's warning about a housing bubble, say, the expert can at least help you to see what could happen if you turn out to be wrong. More broadly, experts can point out risks or opportunities you hadn't thought of and keep you from getting carried away with a single notion like "gold is the only true store of value" or "stocks always outperform."

Roach may or may not be right that the economy has turned less dangerous. But if you've been feeling bearish lately, his analysis can at least keep you from overreacting--by, for example, dumping stocks (which should be a long-term investment) or going deep into commodities (which are expensive now). That's valuable. Because if there's one thing you can predict for sure, it's that hedging your bets is a good idea.