(FORTUNE Magazine) – Just beyond the souvenir shop and down the relic-filled hallways in the Ben & Jerry's factory in Waterbury, Vt., massive pipes pump out 190,000 pints of ice cream each day. Throughout the day, tractor trailers pull up, pick up the mountains of Cherry Garcia or Chunky Monkey pints, and deliver them to depots. From there, the ice cream is shipped out to 50,000 grocery stores in the U.S. and 12 other countries. Once it's on the freezer shelves, it has the magical effect of curing a broken heart, satiating a sweet tooth, or possibly just heightening someone's cholesterol level.

For Ben & Jerry's, that's only the start of the journey. At the company's headquarters in Burlington, Vt., just miles from where the founders opened their first shop 25 years ago, the life of each pint--from ingredients to sale--is tracked. Once the pint is stamped and sent out, Ben & Jerry's stores its tracking number in an Oracle database; then it puts it under the microscope. Using software from a company called Business Objects, the sales team can check to see if Chocolate Chip Cookie Dough is gaining ground on Cherry Garcia for the coveted No. 1 sales position. Down the hall in the marketing department, the company checks to see whether orders online require Ben & Jerry's to make a donation to one of its philanthropies. The finance people pop the number into their journals to show results. Since they started using the software, they've sharply cut the time it takes to close their monthly books. And probably most important to a company focused on customer loyalty, the consumer affairs staff matches up the pint with the 225 calls and e-mails received each week, checking to see if there were any complaints, and if so, which supplier's milk, eggs, or cherries didn't meet the company's near-obsession with quality.

Ben & Jerry's may cultivate a down-home image, but as a unit of $47-billion-a-year Unilever, it depends just as heavily on the stats for its success. And to get those figures, it relies on so-called business intelligence, or BI, software: a plain-vanilla name for programs that crunch huge quantities of data in search of trends, problems, or new business opportunities.

"Huge" may be an understatement. In the 1990s companies spent billions of dollars installing giant IBM, Microsoft, and Oracle databases and data warehouses and then shelled out more on hiring consultants to link the software with just about every bit flooding into the company, from factory inventories to HR information to sales leads. Large retailers typically now have 80 terabytes' worth of information on their products--equivalent to 16 million digital photos or 320 miles of bookshelf. Sears has around 70 terabytes (70 million megabytes) of data, Kmart more than 90, and Unilever 106 terabytes in North America alone. And still that's relatively puny: At the end of last year, Wal-Mart had 285 terabytes in its data warehouse.

But Wall Street doesn't give out awards for collecting data; what the companies needed was a way to put the information to work. Otherwise, says Rebecca Wettemann, vice president of research at Nucleus Research in Wellesley, Mass., "it's like having a bank account with millions of dollars in it but no ATM card. If you can't get it out and can't make it work for you, then it is not really useful."

So in the past few years companies have started demanding ways to make sense of what they've been collecting. Today BI is one of the few sunny spots in a dreary market for technology. According to Forrester Research in Cambridge, Mass., 44% of companies at the end of 2002 were considering buying BI software this year. Merrill Lynch, in its December survey of CIOs, found that the software topped the list of tech spending for 2003. Overall, the market for BI is expected to grow from $4.7 billion this year to $7.5 billion in 2006, according to brokerage house A.G. Edwards.

In perhaps the surest sign in Silicon Valley that a market is ripe, Microsoft announced plans that indicate it's ready to take it over. The company said in mid-February that it would soon integrate BI reporting features into its popular SQL Server database: a major step toward enabling even the most technically illiterate users to parse information using Excel and other Windows-based tools. The biggest players in BI all insist that there's enough market for everyone. And so far the numbers bear that out. SAS Institute, the $1.2-billion-a-year privately held software giant in Cary, N.C., has seen a 6% hike in its revenue in the last year. Cognos, a $491 million company in Ottawa, and Business Objects, a $454 million company in San Jose, have both seen their revenues rise over 30% in the past two years.

While the concept of BI is hardly new, most of the companies and people using it are. Industrial giants like GE and Procter & Gamble have been slicing and dicing their statistical data for decades; now pharmaceutical manufacturers, retailers, universities, and other institutions are demanding the software. Part of the interest comes from increasing pressure on CIOs to deliver on the investments they've made in systems in the past few years. BI software is seen as an attractive way to do that. The software is relatively inexpensive--it typically costs between $35,000 and $75,000, though large enterprise installations can cost millions--and easy to set up. Ben & Jerry's needed only a few weeks to install its system, whereas a large database project can take years.

The results can also show up quickly. Red Robin Gourmet Burgers, a 196-location chain in Greenwood Village, Colo., purchased software from Cognos to track everything from marketing promotions to which state's diners order the most chicken burgers (Oregon, by the way). The restaurant soon found it was wasting $10,000 on unused Alfredo sauce; it has since switched from buying the white sauce to making its own. The Sesame Workshop uses the same software to predict how Elmo dolls will sell on the shelves. And this holiday season, in the midst of launching its latest line of clothes at 183 Sears stores, retailer Lands' End used Business Objects to monitor sales store by store, helping cut back orders to a third of the previous year's amount. "When sales are not growing hand over fist, companies are looking for ways to cut costs and find unmined opportunities," says Ed Maguire, a software analyst with Merrill Lynch.

The software is also allowing employees all around companies to, in effect, run their own investigations. Until now diving through data was reserved for experts--technologists or statisticians who would generate massive reports that would be handed out during meetings, flipped through, then shelved. BI companies now promise that anyone can use the technology. Not that this stuff is simple: The business intelligence software has to worm its way through the entire data center to root out patterns. Typical BI applications first pull information out of giant databases into so-called data marts--smaller clusters of similar information that can keep financial data in one area, inventory data in another. Then the software is ready for the hunt. When a Lands' End product manager wants to know, say, what the hottest-selling turtleneck has been in the last month, the BI software first runs the request through a so-called semantic layer, which translates the query into database-speak (at Lands' End, white turtlenecks go by the handle "66780," for example). Then it uses the terms to gather the relevant data from the right data mart, organize it, analyze it, translate it back, and offer an answer.

The employee sees only the last part. Just about every company now offers stand-alone or browser-based software that the industry calls "dashboards," which presents graphical displays of inventory levels, sales info, and other urgent gauges of day-to-day business.

At Staples, the $10.7-billion-a-year office-supply store, BI recently moved into the mainstream. When Alan Gordon, director of sales forecasting, joined the company in 1993, the chain had 150 stores. His bosses asked him to determine where the company should build new locations--not for his real estate skills (which were zero) but because of his background in statistics. Using programs from SAS, Gordon created a modeling system that takes in 40 types of data, from locations of competitors to sales tax by zip code, and spits out ideal expansion sites. Of the almost 4,000 sites a year Gordon evaluates, about 100 are targeted for a new store. That has translated into 950 stores since Gordon's arrival, almost all of which were sited using SAS. "You've got to have a pretty good argument if you go against the model," he says.

Now Gordon's data hunting has become common across the company. Upstairs from his office in Staples' Framingham, Mass., headquarters, the finance department has started relying more and more on BI software. For years Marci Lerner, the vice president of finance, ran all of Staples' budgets on Microsoft Excel. In the late '90s she turned to software from Hyperion Solutions, a $492-million-a-year company in Sunnyvale, Calif., that has a speciality in financial reporting.

Within a year of installing Hyperion's Essbase, the finance department realized Staples had been misusing the display space in its stores. The shops had long devoted part of their floor space to desks, file cabinets, and other furniture. It had seemed to make sense: The big products delivered better gross margins than pens and paper. But the BI system revealed that the strategy was a mistake. "We found that when you factor in all the costs of storage, distribution, handling, damage, labor, rent/occupancy, etc., the overall profitability of the category is much less than other less bulky, less space-intensive categories," says Lerner. "A desk also takes more labor to sell because you need someone who actually knows a little more about it."

Since then, Staples has shrunk the furniture department in most stores and eliminated it altogether in others. More room for things like Sharpies, yellow pads, and Avery labels helped the company keep its net income growing at a 12% compounded rate over the last five years. The same software led the team to cut back on the floor space allotted to displaying PCs. Now 400 locations stock less computer inventory but offer build-to-order options.

Staples also used BI to reality-test the recommendations of consultants. One who studied Staples stores in 2001 reported that locations did better when they kept computers and other technology on the left side of the store. "Everyone was wondering if we had discovered something new about our customers' behavior," Gordon says. But before starting a 1,100-store overhaul, he ran some BI queries. His finding? There was nothing to back up the consultant's insight.

With BI now accepted outside the tech department, the software companies are trying to push even deeper into corporations. In the past few months they've started promising software packages--often called business performance management or corporate performance management--that will tie the data in to the best practices of the industry or peg it to specific company goals. The idea is to offer executives and their teams quick ways to fix their business. How? That's not entirely clear. Warns Nate Root, a research analyst at Forrester: "What the market is trying to do is put a business spin on BI rather than a technical spin. BPM is just using BI tools to try to run your business better." Part of the confusion comes from the fact that BPM and CPM are not products, but rather umbrella terms, says Frank Buytendijk, a research director with tech analyst Gartner. "We sometimes hear vendors say that CPM is the next big thing after BI. That is nonsense."

In the meantime, companies are using BI not just to solve problems but also to eliminate false assumptions. Last spring Ben & Jerry's noticed a swell in complaints from Cherry Garcia ice-cream-pop customers, most of whom were irritated that the product had too few cherries. The company matched the complaints against the shipment records and analyzed them using Business Objects. First it eliminated the chance that this was a regional problem--complaints were coming from all over the country. Then it queried information on the manufacturing process; the recipe and ingredients all turned out to be normal. Finally, after nixing just about every possibility, the company discovered the problem: The photo on the ice-cream-pop box was not of the ice cream but of frozen yogurt--a product more laden with cherries than the paler-pink ice-cream treat. Ben & Jerry's changed the image on the box, and the complaints melted away.