(FORTUNE Magazine) – Buy and hold. Diversify. Put your money in index funds. Pay attention to the one thing you can control--costs--and keep them as low as possible. Today that is pretty standard, if often unheeded, investment advice. Forty years ago it was revolutionary. The revolution started on college campuses, in particular at the University of Chicago, and it went by the unrevolutionary-sounding name "efficient markets."

"In an efficient market," wrote Chicago professor Eugene Fama in a landmark paper he delivered at the 1969 annual meeting of the American Finance Association, "prices 'fully reflect' available information." That is, in an efficient market you can't beat the market unless you have inside information. So why bother trying?

That logic led, among other things, to the creation of index funds that aim to mimic, not beat, the likes of the S&P 500 and the Wilshire 5000. Today such funds account for about 10% of total U.S. stock market capitalization, as well as 60% of what little money has flowed into equity mutual funds so far this year. But millions of small investors have continued to ignore the advice derived from efficient-markets theory, preferring instead to trade stocks and pile in and out of mutual funds in search of elusive market-beating returns (blowing much of their money on fees and commissions in the process).

Meanwhile, back on campus, a new generation of finance professors has been ripping Fama's teachings to shreds. The organizing principle for this new breed of scholars is not efficient markets but something called behavioral finance. Behavioral finance teaches that stock market investors are irrational, that future stock price movements are at least partly predictable from past behavior, and that careful analysis of past trends and financial reports can pay off. Which happens to be the way most investors see the market already.

Over the next few pages we're going to take you on a journey through the academic battles that have brought us to this point. This is FORTUNE's annual Investor Guide, not The Chronicle of Higher Education, so we wouldn't tell this story if we didn't think it had relevance for investors battered by the experience of the past few years. The message that the behavioral finance guys have for investors is that yes, you can beat the market, but--for reasons that are essential to the whole behavioralist case--you almost certainly won't. As a result, they end up offering much of the same investment advice that the efficient markets folks do. Only this time we might actually listen.

Gene Fama, it must be said, doesn't buy any of this. It's a cold, gray mid-autumn day in Chicago, but Fama is wearing a loud shirt with too-short sleeves that prove him to be probably the buffest 63-year-old finance professor on the planet. With the hint of a Boston accent that remains after two-thirds of a lifetime in Chicago, he's talking the academic version of trash. "I don't know that it's progressed much beyond the level of curiosity items," Fama says of the research done by the behavioralists. "They've got lots of interesting curiosity items."

For years Fama was more than tolerant of this search for "curiosity items." Unlike his longtime Chicago colleague Merton Miller, who saw the behavioralists' work as an ideological assault on free markets (Miller died in 2000), Fama always encouraged empirical research, even research that revealed seemingly inefficient market behavior. In a 1991 sequel to his famous "Efficient Capital Markets" paper of 1969, he acknowledged that reality had in fact turned out a lot messier than he and other efficient-markets theorists had envisioned--although not so messy that the theories couldn't accommodate it.

In 1997, though, Fama wrote still another paper, one that argued that a lot of the purported market anomalies discovered by the behavioralists were due to bad statistical work, and that the behavioralists' attempts at building a theoretical alternative to the efficient-markets hypothesis had so far been "embarrassing." The paper was something of a sensation--until interest in the D.C.-area sniper sent a piece called "Multiple-Victim Public Shootings" to the top of the charts in October, it was the most downloaded work in the almost six-year history of the Social Science Research Network, a leading academic website. Fama had no problem with behavioral finance as a critique of efficient-markets theory. But he thinks it's a disaster as a replacement. "I don't know what asset pricing would look like in a world that really took behavioral finance seriously," he says. "If you really think prices are incorrect, what are you going to tell me about the cost of capital?"

We'll get back to that question. But first let us document the behavioralists' triumph. Half of this year's economics Nobel went to their patron saint, Princeton psychologist Daniel Kahneman (the other half went to Vernon Smith of George Mason, whose economic experiments have also shot holes in efficient-markets dogma). Then there's the John Bates Clark Medal, awarded by the American Economic Association every two years to the most important U.S. economist under 40: The 1999 and 2001 editions both went to behavioralists. On the pop-culture front, Yale efficient-markets skeptic Robert Shiller's 2000 bestseller Irrational Exuberance was the most talked-about book by an economist in years.

The most dramatic development of all, though, may be that the office directly below Fama's at Chicago's Graduate School of Business now belongs to behavioralist pioneer Richard Thaler, 57. A magazine profile last year characterized Thaler, to the undying amusement of his students, as "thick-set," but that's not quite fair. He is not the jock that his upstairs neighbor is--Fama beats him at tennis. But Thaler, who arrived in Chicago in 1995 after years in the relative academic wilderness of Cornell University, appears to have eclipsed Fama as the most influential faculty member at the business school that has had more influence on the study of finance than any other.

For decades the University of Chicago was the temple for those who believed that markets always got things right. Thaler's intellectual journey, in fact, began in the early 1970s on a campus that followed Chicago's example to almost absurd lengths: the University of Rochester. "I can remember my professors yelling, 'The price is right! The price is right!'" recalls Dartmouth finance professor Kenneth French, who arrived at Rochester just after Thaler left. "It was like a bad game show."

Rochester was an extreme version of the direction the entire economics profession--not just its finance offshoot--had taken. The idea that economic activity can be explained as rational individuals trying to maximize their wealth goes back at least to 18th-century Scotsman Adam Smith, and the thought that all this was best expressed mathematically occurred to economists as long ago as the 1870s. But the approach didn't really take off until the 1947 publication of MIT professor Paul Samuelson's Foundations of Economic Analyses, which recast the principles of economics in the language of Newtonian calculus. The mathematization of economics that followed swept all before it. While academic economics had previously allowed room for multiple streams of thought, it took just a couple of decades before mathematical models built on the assumption of rationality were the only game on campus.

One of the most compelling of those models, and the one that seemed most closely to fit real-world data, was the efficient-markets hypothesis. It had its roots in empirical research that appeared to show stocks moving in a random walk--albeit with an upward trajectory determined by rising corporate earnings. The theoretical explanation, first proffered by Samuelson in 1965 and soon elaborated by Fama (who got his Ph.D. at Chicago in 1964) and others, was that stock prices fluctuate randomly because all knowable information about the value of a stock is already discounted in the price. That is, prices change only in reaction to news, which is by definition unpredictable.

For this view of price movements to work, the market has to behave rationally. That doesn't mean every last investor has to be rational; it's enough to assume that the hordes of irrational investors are irrational in different ways, thus canceling each other out. Or failing that--and here the theory begins to wobble--rational investors would be able to take advantage of the market's temporary insanity to make a killing and push prices to where they belong.

That brings us back to Thaler, who was working on a Ph.D. in economics at Rochester in the early 1970s. His dissertation was an attempt to put a value on human life by looking at how much more people were paid to work in risky fields like mining and logging. He was working on the assumption, of course, that people rationally weighed the risk of death in their decision to accept a job.

Along the way Thaler decided to ask a few friends how much they'd be willing to pay to eliminate a one-in-1,000 chance of immediate death and how much they would have to be paid to willingly accept an extra one-in-1,000 chance of immediate death. What he found was that they wouldn't pay much for the extra margin of safety but demanded huge sums to accept added risk--which isn't, strictly speaking, rational. "I came to two conclusions about these answers," Thaler wrote years later. "(1) I had better get back to running regressions if I want to graduate, and (2) the disparity between buying and selling prices was very interesting."

Thaler did discuss his subversive thoughts with a few trusted colleagues and people from other disciplines. One of those people happened to be a newly minted psychology Ph.D., who sent Thaler a copy of a 1974 article by Israeli psychology professors Amos Tversky and Daniel Kahneman. (Tversky died in 1996; if he were still around, he surely would have shared in this year's Nobel.) The article argued that in making decisions involving probability and risk, people rely on mental shortcuts that "are highly economical and usually effective but ... lead to systematic and predictable errors."

It was that last part that was so significant. That people make judgment errors wasn't news, but if those errors were "systematic and predictable," well, that was something an equation-wielding economist could get up and run with. (And making decisions involving probability and risk is what investing is all about, although Thaler wasn't really thinking about that at the time.) Thaler wangled a short-term research gig at Stanford University when Tversky and Kahneman were visiting professors there in 1978 and ended up staying for 15 months. Then he played a key role in unleashing Tversky and Kahneman's ideas on the economics profession.

The profession did not immediately respond with great enthusiasm. But over the years Thaler began to collect a few allies. Some were economists who shared his fascination with psychology. Others were number-crunching finance professors who had stumbled across seemingly irrational market phenomena--from short-lived ones like the "January effect" of rising stock prices in the first weeks of the year, which disappeared not long after people started writing about it, to the persistent tendency of "value" stocks with low price-to-book ratios to outperform the market. Throughout the 1980s the ranks of the doubters grew, but they remained a fringe element.

The efficient-markets guys, meanwhile, not only had come to occupy the academic mainstream but also had moved in on Wall Street. Not surprisingly, their initial relations with the Street had been hostile. What the professors were saying, after all, was that highly paid fund managers and analysts were not worth a dime. Some of the professors clearly reveled in that: In one famous mid-1960s exchange, a money manager asked MIT's Paul Cootner, "If you're so smart, why aren't you rich?" To which Cootner replied, "If you're so rich, why aren't you smart?"

The answer to that second question was that people on Wall Street didn't have to be smart to get rich, since they could make money off fees and brokerage commissions even when their market calls stank. But the devastating bear market of the 1970s caused some investors to question whether the people with whom they'd entrusted their money really were worth the expense. One logical result of such thinking was the index fund, which instead of trying to outsmart the market simply tried to imitate it while charging much lower fees than actively managed funds do. The first index fund for institutional investors was started in 1971 by Wells Fargo Investment Advisors (now Barclays Global Investors) in San Francisco. The first such fund for retail investors--the Vanguard Index Trust--was launched five years later.

Meanwhile a few finance scholars of a more diplomatic bent than Cootner began spreading their ideas of risk and return in the real world. Princeton economist Burton Malkiel's A Random Walk Down Wall Street, published in 1973, probably played the biggest role in bringing efficient-markets thinking to the retail investing masses. But on Wall Street itself, the most important messenger was William Sharpe.

Sharpe, now 68, grew up in Southern California and learned his economics at UCLA. He was of the efficient-markets school, but his work (for which he won the economics Nobel in 1990) appealed even to those who still hoped to beat the market. In an efficient market the only way to outperform the market is to take on more risk. Sharpe devised a simple measure of risk based on past volatility, called "beta," that could be used to build balanced portfolios--and to measure whether active money managers were actually beating the market or just taking on extra risk.

Sharpe wasn't content to make his point merely in academic journals. He wrote textbooks on investments and finance and did so much consulting for Wall Street firms and pension funds that he gave up full-time teaching at Stanford in the mid-1980s. In 1996 he even launched a dot-com, Financial Engines, to make his advice available to small investors. So while Sharpe believes in efficient markets, he has also spent much of his career helping investors make choices. That, it turns out, makes him a big fan of behavioral finance. "As a practical matter, I still think it's prudent to assume that the market is pretty close to efficient in terms of pricing and risk and return and all that," Sharpe says. "On the other hand, we've certainly learned from cognitive psychology that ordinary human beings need to have alternatives framed in ways that can help them make right decisions rather than wrong decisions."

Most of the wrong decisions investors make, behavioral research has shown, stem from overconfidence. That is, we think we know more than we do. We trade too much, we don't diversify enough, and we extrapolate from the recent past to make assumptions about what will happen next.

As a result, much of what the behavioralists have to offer in terms of advice has to do with protecting retail investors from themselves. That's why Thaler spends a lot of his time thinking about how best to design 401(k) plans. It's why Sharpe incorporates behavioralist research into the advice Financial Engines doles out. And it's almost certainly why Daniel Kahneman, when asked by a CNBC anchorman the day after his Nobel was announced in October what investment tips he had for viewers, responded, "Buy and hold."

When I recount Kahneman's words a few weeks later to Fama, he reacts with glee. "That means I won!" he shouts. It is, on one level, an absurd claim. The behavioralists are now clearly the dominant stream in academic finance, having made the leap from outsider status during the 1990s as a new generation of professors rose to positions of prominence. But the real-world phenomenon that cemented the behavioralists' victory also illustrates why, when it comes to actual investing advice, they sound so much like Fama and Sharpe.

That real-world phenomenon was the stock market bubble of the late 1990s. According to strict efficient-markets thinking, there must be a rational explanation for what happened. Fama describes those sky-high Internet stock valuations as a risky but not crazy bet that one or two of those money-losing Net companies would end up as big as Microsoft. But he's almost all alone on this one. "We have just lived through the biggest bubble of all time," says Malkiel, who now calls himself a "random walker with a crutch." Fama's favorite collaborator, Dartmouth's French, is on the verge of using the b-word as well when he stops himself. "I work very closely with Gene," he says. "He would be very upset if I used that word in print."

Yale economist Robert Shiller has no such compunctions about ticking off Gene Fama. In 1984 he declared that the logical leap from observing that stock price movements were unpredictable to concluding that the prices are in fact right "represents one of the most remarkable errors in the history of economic thought." That was Shiller's first brush with fame. He got more popular attention after the 1987 stock market crash, which the efficient-markets professors had trouble explaining. ("It's weird," Sharpe told a reporter at the time. Later his mother called to berate him: "Fifteen years of education, three advanced degrees, and all you can say is, 'It's weird'?")

Shiller is 56 and did his economics training under Samuelson at MIT. He and Thaler have long been allies, but Shiller seems less interested than many of the other behavioralists in assembling the cognitive-psychology building blocks of a market bubble (which would involve that persistent flaw of extrapolating from the recent past to make assumptions about what will happen next). Instead he's perfectly willing to accept at face value the conventional wisdom that markets are sometimes taken over by fads and mass hysterias. By the mid-1990s Shiller had become convinced that we were entering into one of those mass hysterias. His evidence was straightforward: Price/earn-ings ratios were really high. He began sounding the alarm wherever he could, including the offices of the Federal Reserve Board. Then he wrote Irrational Exuberance, which hit bookstores in March 2000, just as the market peaked.

The book's perfect timing was dumb luck, Shiller himself says. And while he took most of his own money out of the stock market in the 1990s, his advice to investors now is to "diversify completely" and not try to beat the market. This happens to be what Sharpe would tell you. Or Fama. Or Thaler. The dirty little secret of the behavioralists is that, for all their work on investor irrationality and market anomalies, they still believe that markets work pretty well and that trying to outguess the collective wisdom of millions of investors is usually futile. In answer to Fama's question of how they plan to calculate the cost of capital in a world where prices are incorrect, the behavioralists say that for the purposes of such calculations, they'll just assume that prices are right.

But efficient-markets theory has a dirty little secret, too, which is that for the market to remain efficient, there have to be lots of rational investors who believe enough in the market's inefficiency to spend their careers trying to beat it. Behavioralist theory, of course, has no problem accommodating the belief that some investors can beat the market. In fact, several behavioralist professors, Thaler included, have money-management firms that try to take advantage of the anomalies they discover in their research.

But there's a limit to the riches that can be dredged from market anomalies. That's because "markets can remain irrational longer than you can remain solvent." This aphorism is usually attributed to economist and speculator John Maynard Keynes, and there are those who contend that the whole of the behavioralist case is contained in chapter 12 of Keynes's 1936 General Theory, with its wonderful depiction of investing as a game of musical chairs. But the argument of modern behavioralists includes a crucial observation that wasn't in Keynes--that professional investors are now under so much pressure from their customers that they cannot make the kind of long-term bets that might beat the market. If they do, as was the case with a lot of value-oriented mutual funds in the late 1990s, they can soon find themselves without any customers' money to invest.

That gets us to a world in which an investor with enough staying power and contrarian gumption can beat the market, but the vast majority of mutual funds and hedge funds don't. In other words, the behavioralists have reconciled the success of a Warren Buffett (which efficient-markets purists have absurdly termed dumb luck) with the overwhelmingly empirical evidence that most professional money managers fail to beat the market.

This is, we posit, a major intellectual accomplishment. What does it mean for you? That's easy: Buy and hold. Diversify. Put your money in index funds. Pay attention to the one thing you can control--costs--and keep them as low as possible.

FEEDBACK jfox@fortunemail.com