(MONEY Magazine) – Many investors think that Standard & Poor's 500-stock index, the most popular benchmark for measuring market performance, is a stable list of the country's 500 biggest companies. Not so. The S&P 500 is, in fact, made up of the 500 stocks that most appeal to seven people who meet once a month on the 44th floor of 55 Water Street in downtown New York City. Like the admissions committee of an elite country club dropping white or black balls into a wooden box, the Index Committee of Standard & Poor's meets in secret; its proceedings are at least as private as those of the Federal Reserve Board, with no minutes released or memorandums issued. But it's become clear that the index keepers are changing the S&P in a radically new and potentially disruptive way. They are systematically tearing out sluggish Old Economy value stocks and replacing them with trendy New Economy names. The implications for investors are huge. One example: The switches made last year hurt the performance of the index by an estimated total of $100 billion.

Vast sums of money ride on the S&P index committee's deliberations. The S&P 500 makes up about 70% of the value of all U.S. stocks; roughly $1 trillion is invested in index funds that seek to track its performance precisely, with trillions more in other funds that shadow it. To sophisticated investors, the S&P 500 is the market.

When the S&P committee replaces one stock and adds another, it issues a terse press release at 5:15 p.m. New York time, after the stock market closes. Only then does an S&P official contact the companies that have been added and deleted. The announcement sets off a huge chain reaction, as the index funds swing into action, buying the new stock in massive volumes. By the end of the first day a stock is included in the S&P 500, roughly 8% of its shares disappear into the portfolios of index funds, where they remain indefinitely. Non-index funds load up on a newly minted member as well, betting that the stock's price will rise now that it has joined the ultimate honor roll for American companies. Meanwhile, the shares that the committee deletes from the index drop that day like ducks shot out of the sky.

"The idea in running the index," explains index committee chairman David Blitzer, "has always been that it should reflect the stock market and, through the market, the U.S. economy as a whole." Blitzer, an affable and refreshingly forthright fellow who looks like a blend of an Amish farmer and a rabbi, is S&P's chief investment strategist and the author of a solid new book on index funds, Outpacing the Pros.

Beneath the surface, however, the S&P 500 is in tectonic turmoil. The index always contains 500 stocks, no more and no less, so a new company can enter the list only by dislodging one that's already there. Historically, new stocks have joined the list through attrition, as existing members have been acquired, merged or gone bankrupt--what S&P calls nonvoluntary changes. To be sure, the index committee used to make the occasional judgment call, kicking out a stock whose value had shriveled to almost nothing. But from 1990 through 1994, S&P booted off only 10 stocks for "lack of representation"--code for "we don't want you in our club anymore."

Over the past few years, however, S&P's committee has been increasingly twitchy. Starting in 1995, it has evicted old stocks from the S&P 500 and stuffed in new ones at an unprecedented pace (see the bar chart on page 85). And last year, the committee threw out 18 companies--more than one of every 30 stocks in the entire index--for the dreaded "lack of representation" and fallen market value. That's the highest annual total in the index's 75-year history (see the graph at right).

Of the 18 companies added, more than half were technology stocks. What's more, even in its nonvoluntary changes triggered by mergers and the like, S&P has been using its discretion to plug in New Economy replacements. By the time tech stocks peaked in March 2000 at 33.47% of the S&P 500's entire capitalization, just under two-thirds of that tech total came from stocks added since Jan. 1, 1991, including such lesser-known outfits as Mercury Interactive and Xilinx.

Largely as a result of this rejiggering, historical comparisons using the S&P 500 have become an exercise in juggling apples and oranges. True, the index now trades at roughly 24 times earnings, far above its historical average of 15. But the old average reflects a period when the index committee wasn't packing the 500 with high-priced stocks. How can we tell if the market is overpriced when "the market" is different?

Then there's the impact these changes have on the managers of mutual funds and hedge funds, whose reputations--and bonuses--are often tied to beating the index. If they want to be rewarded by their bosses, and by investors, they need to respond to the S&P committee's adjustments by juicing up their own portfolios. It's a lesson that fund manager Bill Miller of Legg Mason Value learned well during the '90s, as he shifted his portfolio toward growthier and growthier stocks, making a name for himself by repeatedly beating the index. Other, less flexible value managers found themselves out of favor and, sometimes, out of work. Meanwhile, we investors have continued to apply the S&P 500 as a universal benchmark rather than the growth-stock index it has become. In the process we may well have distorted the shapes of our own portfolios.

THE LAST SHALL BE FIRST

In many ways the changes to the index have been backfiring on all of us whose investment results are tied to the health of the market as the S&P 500 measures it. First of all, the companies that S&P has added to the 500 index often perform worse than the ones it has thrown out. I asked Aronson & Partners, a Philadelphia-based investment firm that manages $5.2 billion, to analyze the performance of the voluntary substitutions S&P has made in the index since 1997. Looking at the 12-month periods before the committee made its changes, Aronson found that the average stock that was added (and is still trackable) had gone up 65.75%, while the average stock that was deleted had fallen 11.97%. Over the next 12 months, the typical newcomer went on to lose 47.66%, while the average exile gained 16.71%. These numbers are not adjusted for the market's overall return, but if you view the addition of a stock to the index as a "buy" and a deletion as a "sell," the S&P committee has lately been buying high and selling low.

The voluntary changes made to the S&P 500 in the year 2000 alone, reckons Aronson & Partners, reduced the index's return by 0.84%. That may sound like chicken feed, but for a $12 trillion market, it amounts to $100 billion.

A big chunk of that immense loss can be traced back to last July, when the S&P committee threw out Rite Aid Corp., one of the nation's largest drugstore chains, and replaced it with JDS Uniphase, the fiber-optics giant. JDSU soared 27.3% on the news, to $126.6 billion--the biggest stock ever added to the S&P 500. Poor Rite Aid, meanwhile, lost 10.1% on the news of its departure and left the index with a value of just $1.1 billion. In one fell swoop, JDSU raised the total value of the S&P 500 by more than $125 billion and diluted the influence of every other stock in the index. The aftermath was ugly. JDSU has tumbled 84.4%--and Rite Aid is up 104.2%.

Probably the most ironic substitution of all occurred last June, when Agilent Technologies, the equipment maker spun off from Hewlett-Packard, elbowed into the S&P 500, knocking aside Nacco Industries, an Old Economy grab bag of coal mining, forklift trucks and Hamilton Beach and Proctor-Silex kitchen appliances. In market value, Nacco was then dead last in the index at just $300 million. "When you rank near the bottom of the S&P by market cap," says Nacco investor-relations manager Ira Gamm, "you just know you're on their hit list." (Blitzer doesn't deny this; as he puts it, "Stocks No. 497 or 498 or 499 are generally the ones our analysts are nervous about.") Agilent shot up 18.9% as it went into the index, while Nacco dropped 15.4%. It just so happens that Nacco's chairman and CEO Alfred Rankin serves on the board of directors of Vanguard's Index 500 fund; he had no choice but to sit by while the fund replaced his own stock with Agilent. The ironies don't end there: Agilent has since fallen by 53.9%, while Nacco is up nearly 103.1%.

A SUBJECTIVE PURGE

Blitzer denies that S&P has a master plan to pack the index with technology and growth stocks. "I don't really think we're liable to that charge," he says. "We just followed the show." As tech stocks boomed in the late 1990s, S&P's mandate to reflect the total market gave the committee no choice but to add more tech; otherwise, investors would have cried, "Dinosaur!" Blitzer insists that "outperformance is not at all one of our criteria. We don't say, 'This is a hot stock.' We say, 'This is a leading company in a leading industry.'"

And S&P does have quantitative criteria: a stock should be valued at a minimum of $3 billion to $5 billion, at least 50% of its stock should be public, no less than a third of its shares should trade regularly and its industry's weight in the index should be similar to its weight in the total stock market.

Yet Blitzer makes no bones about how subjective S&P's decisions can be. He admits that the committee used last year's crash as a pretext to purge some of the most sluggish Old Economy stocks (like Bethlehem Steel, Owens-Illinois and Polaroid). "There are a lot of stocks in the 500 that if they weren't in already, wouldn't go in," says Blitzer. "There's a reluctance to do a big housecleaning, though every once in a while there's a little of that." For example? "If a company's very, very small and nobody [on the committee] thinks it's going anyplace, or if, like steel, a whole industry seems to be getting left behind." In the end, despite Blitzer's assertions to the contrary, it's hard not to feel that S&P's moves are driven at least partly by a desire to make room for what's hot.

WHAT'S THE MANDATE?

So how do the folks at Standard & Poor's feel about the carnage among the stocks they've added to the index--and about the stellar returns among the stocks they've removed? "It's a disappointment," says Blitzer, laughing at the sound of his own understatement, "but I don't think that it impugns what we 're doing." After a pause, he adds, "There's not an excessive amount of Monday-morning quarterbacking here. I can't think of anything where the index committee sat down a month later and said, 'We really shouldn't have done that.'"

Blitzer says that in 1999, "especially with Amazon," S&P hotly debated relaxing its rule that companies must have positive earnings (or have had them recently) to be eligible for the index. The rule remains, at least for now. But insisting on earnings is one thing; insisting on reasonable stock prices in relation to earnings is another. "We get accused of being crazy on [price/earnings ratios]," says Blitzer, "but we don't pass judgment on valuations. P/E is not one of our criteria." It's an intriguing comment for the keeper of an index that for most of us is all about investing.

To be fair, Standard & Poor's is mainly trying to ensure that the 500 index lives up to its mandates of reflecting the overall stock market and providing a benchmark of leading companies in leading industries. But at the same time, S&P has altered the index so aggressively that, as I see it, chasing higher performance seems to be part of the mandate as well.

So what should you make of all this? One thing that's clear is that you have to think about the S&P 500 differently. You shouldn't abandon index funds; they remain the cheapest and most reliable way to capture the maximum return on stocks in the long run. But you need to realize that the S&P 500 is no longer a a good yardstick for value stocks and funds, and that it's rapidly becoming useless as a tool to compare today's market valuations with those of the past. Finally, it's far from clear whether Standard & Poor's remarkable reconstruction of the 500 index will turn out to be a good idea or a disaster. Like it or not, we're all stuck going along for the ride.

ADDITIONAL REPORTING BY ANDREA BENNETT