(FORTUNE Magazine) – Tod Nielsen and his lovely wife, Allison, were at the front door of their home in Redmond, Wash., greeting guests. Tod was in a sleek one-button tuxedo with silver-and-white bow tie, while Allison was wearing a drop-dead-gorgeous black-and-white gown. It was Oscar night. On each side of the door was a ten-foot-tall statue of Oscar. In the dining room the buffet included a delicious pesto cream cheese in the shape of Oscar. And on a table just inside the entranceway was a thick sheaf of ballots so that all the guests could vote on the Big Six categories.

Yes, it was Tod and Allison's annual black-tie Oscar party. They've thrown it for the past seven years--"just because it's fun," explains Allison--and it's become quite the tradition. This year, though, there was a twist. Along with neighbors, colleagues, and other friends of the Nielsens', I was there, too, conducting a little economics experiment. No, seriously.

I first met Tod in 1998, when he was a high-ranking Microsoft executive and I was covering the anti-trust trial. (He's now chief marketing officer at BEA.) A friendly, funny, gregarious soul, the 33-year-old Nielsen was one of the Microsoft hands we reporters most enjoyed spending time with; though he never veered from the company's party line, he always had a twinkle in his eye. At the Christmas bash the press threw for all the trial participants, Tod handed out seat cushions to commemorate the long hours we'd spent sitting in the courtroom. When the Oscars rolled around in 1999, Tod and Allison invited me to their second annual party. I hauled my tux to the Great Northwest and had a grand time. Everybody voted on the winners, and then we all sat around one of the Nielsens' three big televisions, sipping champagne and rooting for our favorites.

About a year ago I had lunch with Tod when he was in New York City on a business trip. He was in fine fettle, as always, and at some point during the lunch--it's quite possible we'd had some wine--I broached my idea. Could I come to the next Oscar party and monitor the voting? I wanted to test the so-called wisdom of crowds. After a quick consult with Allison, Tod said he thought it was a swell idea.

Have you heard about this wisdom-of-crowds thing? It's been gaining popularity lately, championed by, among others, a Columbia University Business School professor named Paul Johnson, and mentioned a number of times in print by New Yorker columnist James Surowiecki. The central thesis is that there are circumstances when "collective intelligence"--that's Jim's preferred phrase--will arrive at a more accurate conclusion than any single individual can. There is no hard science to back this up, I hasten to add, but it's darn interesting. According to Jim, who has a book coming out on the subject, the theory helps explain why the best lifeline on Who Wants to Be a Millionaire?--correct an astounding 91% of the time--is the answer from the audience. That's why when people are asked to guess the number of jellybeans in a jar, the mean answer is likely to be far closer to the correct one than any individual answer. Bookies use collective intelligence to set odds. In his book, Surowiecki gives lots of far more serious--and in some cases jaw-dropping--examples, including one in which a naval officer located a sunken submarine by asking individuals with narrow areas of expertise to take their best guess--and then plotting their collective answer. That answer turned out to be only a few hundred yards from the sub.

The wisdom of crowds is also an intriguing way of looking at the stock market. As Cornell economist Maureen O'Hara has said, "While markets appear to work in practice, we're not sure how they work in theory." The wisdom of crowds is one plausible theory. Consider: Each of us knows very little, really, about the companies we invest in--certainly not how they're going to perform in the future. Yet our collective buying and selling often sends stunningly accurate signals about a company's prospects, which are reflected in the stock price. "When our imperfect judgments are aggregated in the right way," Jim concludes, "our collective intelligence is often excellent." Well, okay, not in the middle of a bubble--that's the madness of crowds--but most other times.

In 1995, Paul Johnson started using the Oscars to test the theory. That year--and on four subsequent occasions--he had his students vote on 12 Oscar categories, including editing and cinematography, on which the students were unlikely to have well-formed judgments. His results over the years have been pretty consistent and pretty remarkable. In all but one year, the consensus tally--that is, the Oscar nominees with the most student votes--has never lost to an individual student (though some students have tied the consensus). "The second surprise," wrote Paul in a little monograph he shared with me recently, "is that the consensus did remarkably well despite the low score of the average student." For instance, in 1997--a year in which 125 students voted--the average student was right on only 4.83 predictions. But the consensus got 11 out of 12 correct! Even in 2002, the one year when some students beat the consensus, the five consensus winners still beat the student average of 3.60. For whatever reason, says Paul, "the consensus appears to be a powerful force."

So, back to the Nielsens' Oscar party. Let me say first of all, Tod and Allison, you hosted a wonderful party. Great wine, delicious food, lots of interesting people. I really wanted to stay longer, and if I hadn't been so jet-lagged, I would have. But I wound up thinking, at least at first, that I'd picked a lousy year to use your party for testing the wisdom of crowds. Allison said afterward that never before had a guest picked all the Big Six winners. This year, however, all the odds-on favorites won, and when Allison was finished adding up the ballots, 12 of her guests had batted 1.000. Needless to say, the consensus got them all right too. Interestingly, it also crushed the average black-tied voter, who got only 4.2 winners, thus replicating one of Paul's main findings.

Later, as I was trying to interpret the results, I spoke to both Surowiecki and Johnson. Jim agreed with me that this was not a good year to test the wisdom of crowds, simply because there were such clear favorites. But Paul took a different view. He had tested his usual 12 categories this year, and his consensus had gotten all 12 right. The average student, by contrast, had only seven correct predictions. "It almost does not matter whether the year was an 'easy' year to predict or a 'crazy' year that was hard to predict," he wrote me in an e-mail. "The consensus always does better. I think the takeaway is the emergent power of the consensus."

Let me tell you one other thing that happened, though. Every year Allison includes a few extra questions in case she needs a tiebreaker. This year her questions were: When were the Oscars first broadcast in color? (1966.) What is the weight of an Oscar statue? (8.5 pounds.) And what is the height of an Oscar statue? (13.5 inches.) I decided to tally those guesses as well.

Here's what I found: The mean answer to the first question was 1963.5. Not so great, I thought; fully 23 of the 64 partygoers were closer than that, including five who said 1966. Then the weight: The consensus answer was 6.2 pounds. Again, I thought, underwhelming. Discouraged but undaunted, I tallied the consensus answer for the height. It was...(drum roll, please)... 13.54466--uncannily close to the real answer of 13.5! And it got that close without a single person at the party filling in the correct response.

"Wow," said Jim when I told him that. But a more skeptical Paul wrote, "I'm impressed with the height question, but perhaps that's just data mining."

Clearly, this entire subject deserves further study. I plan to reopen my investigation next year at my laboratory in Redmond. I'll have my tuxedo cleaned and pressed by then.

FEEDBACK jnocera@fortunemail.com