(FORTUNE Magazine) – Rat No. 3 is determined. He lumbers straight to an octagon in the corner of the pen and runs his white whiskers over its edges. In another corner is a white ball. It holds little interest for No. 3--exactly the result Merck researcher Rebecca Dias is hoping for. The rat has a few micrograms of Merck's memory-enhancing compound coursing through its tiny veins, a prototype medicine that might someday treat Alzheimer's patients or people who have suffered a stroke. How does Dias know the serum is working? Rats can only remember what's happened to them in the past four hours. No. 3 spends more time investigating the octagon, says Dias, because he recalls seeing the ball the previous day--and he's bored with it.

The medicine Dias and her team are testing, at Merck's Terlings Park labs just outside London, still has some seven years of lab work and clinical trials to go before the company can send it to the FDA for approval. The odds that any test compound will make it to market are incredibly low. It's a fact Dias and her colleagues are painfully aware of. At 29, and at the beginning of her career with Merck, Dias will be lucky if she sees one test compound reach pharmacists' shelves before she retires. Even so, she is excited by her findings. "It's so novel we can't really talk about it right now," says Dias. "But imagine if you were able to turn a 70-year-old's memory into that of a 50-year-old."

Unfortunately, Merck investors don't need a pill to remember happier times. Back in 2000, the Whitehouse Station, N.J., behemoth's revenue growth was tops in the drug industry. Between 1994, when Ray Gilmartin got the CEO job, and 2000, Merck's annual earnings growth averaged 17%; in the 1990s, Merck's shares rose 600%. But those days have faded. Following a raft of patent expirations on some of Merck's biggest drugs, the $48-billion-a-year company will have no earnings growth at all in 2002. Since December 2000, the stock has fallen 43%, vs. 21% for the S&P; Merck shares recently traded for $53 each.

What's odd about Merck's predicament is that its scientists have been demonstrably more innovative than the competition. Since 1996, Merck researchers have patented 1,933 new compounds, 400 more than second-place Pharmacia. Certainly not all newly discovered compounds lead to viable products, but patents are a prime indicator of research productivity. Even more impressive, Merck's scientists do more with less. Their discoveries cost an average of $6 million per patent, the lowest in the industry. Competitors like Pfizer, Eli Lilly, and Pharmacia plow about 20% of drug revenues back into R&D, while Merck reinvests just 12%.

Exceptional science is nothing new for Merck. The company traces its roots to 1668, the year Friedrich Jacob Merck took over the Angel Drugstore in the German town of Darmstadt. By the 1860s, Merck was the most powerful drug firm on the Continent, the first to commercially produce morphine, codeine, and cocaine, then considered medicinal. Thirty years later scion George Merck crossed the Atlantic to build a North American plant in Rahway, N.J., where the company still maintains a research facility. In the 1930s, Merck led the industry in synthesizing vitamins like B-1 and B-12, and in 1949 it worked with Nobel Prize-winning biochemists Edward Kendall and Phillip Hench to produce the first commercial batch of arthritis-reliever cortisone.

That focus on breakthrough medicine is the main reason Merck continues to attract the world's leading biochemists and engineers--people like Dennis Choi, former head of neurological research at Washington University in St. Louis. Last spring he left the cozy confines of academia to join Merck as executive vice president of neuroscience. "I was having a ball in St. Louis; I had tenure. But I came here because Merck enjoys a widespread reputation for sustained application of the best science," says Choi. Does he feel pressure to produce a big drug? Sure he does. "But this is a singular moment in medical history," he says. "Things are definitely going to happen."

The question is when. Despite the company's edge in innovation and efficiency, Merck hasn't been able to churn out enough new products to offset shrinking sales from drugs losing patent protection. The malaise isn't unique to Merck, of course. Beset by cutthroat competition from generic drugmakers, pricing constraints imposed by managed care, threats of government price regulation, and more intense FDA scrutiny, most brand-name drugmakers are struggling. Merck's problems are complicated by its decision last winter to spin off Medco, the prescription-benefits-management firm that provided more than half of Merck's revenues. As a result, the company's future depends more than ever on blockbuster drugs--at a time when the entire pharmaceuticals industry is having difficulty delivering them.

That's been the case for a while now. During the 1990s only 15% of the 1,035 major drugs approved by the FDA used new active ingredients or treated illnesses in original ways. The majority of approvals were modifications of existing medicines, so-called me-too drugs. Even without much innovation, the cost of ushering a new medicine to market has risen 60% since 1990, to more than $800 million.

In this difficult environment, a drugmaker's pipeline of soon-to-be-released medicines is critical. Patents give companies exclusive sales rights for an average of 11 to 12 years, but once that period expires, generic drugmakers enter the market and begin selling the medicine at cut-rate prices. Within a month or two the name-brand company can lose as much as 80% of its sales to generic replacements. In order to stanch that revenue drain, name-brand companies must generate a constant flow of new products. Yet their pipelines are just about tapped out.

No company illustrates Big Pharma's woes better than Merck. Last month the patent on its $1.1 billion Prinivil ran out, ending a spate of recent expirations that included Mevacor, Vaseretic, and Pepcid--together representing annual sales of roughly $4.5 billion, or 10% of Merck's total. Another potential weak spot is Merck's Prilosec, a heartburn remedy developed in a joint venture with AstraZeneca PLC. The world's second-bestselling drug, Prilosec garners $3 billion a year in U.S. sales, a third of which goes to Merck. But the FDA has just awarded a generic maker the right to market a form of the drug. Merck and AstraZeneca are fighting to extend the patent.

When Merck executives talk about promising new drugs, they point to Arcoxia, an arthritis pain reliever; Zetia, a cholesterol medicine; and something the company calls Substance P, a drug to prevent nausea and vomiting in chemotherapy patients. The company is loath to mention, though, that it pulled its FDA application for Arcoxia last spring for additional review, and that Substance P has run into snags in clinical testing. The release dates for both have been pushed back. Even so, Merck claims its pipeline beyond those drugs is among the strongest ever, with future medicines like a diabetes drug called KRP297, a human papilloma virus vaccine, and an HIV vaccine that is in phase I of clinical testing.

Wall Street isn't convinced. Says Robertson Stephens analyst Robert Hazlett: "Are these new drugs enough to keep substantial growth? No. Until we see proof, there's going to be skepticism."

The disbelief sounds awfully familiar to Merck's old guard. Ed Scolnick joined the company in 1982 as a research scientist and achieved a place in the Merck pantheon when he led the team that developed the cholesterol reducer Mevacor in the mid-1980s. Now executive vice president of science and technology, he's seen Merck endure many feast/famine cycles. "We went through the same stuff in '94 and '95," Scolnick says. "I don't think most analysts get it. We told them about Fosamax, Singulair, and Cozaar. They didn't think these drugs would be big. They said we had no pipeline." Those drugs, it turns out, are three of Merck's biggest sellers. Scolnick adds: "We know what we have, and we feel pretty confident. Understate and overperform--that's how I was brought up in science."

Ray Gilmartin is fond of penciling charts and diagrams to add texture to his soft-spoken words. He's already used two sheets of paper to illustrate the lay of the land in the drug industry, but now, in his private meeting room, he pulls the yellow pad closer to the edge of the table and draws a bold black line from the top left-hand corner of the page to the bottom right. That dark downward stroke symbolizes 2001 sales of Merck's osteoarthritis drug, Vioxx, he says, and helps explain the company's no-growth fix. Three years ago Merck introduced Vioxx, expecting it to hit big--and it did. But just as Vioxx was entering its second year of sales in August 2001, the Journal of the American Medical Association published a report that questioned whether Vioxx and drugs like it could be linked to heart attacks. Sales dipped immediately. "It was a one-time, unexpected event," says Gilmartin. Slowing Vioxx sales, he says, were the principal reason he was forced to revise the company's earnings forecast for the year.

Critics say Gilmartin could have done more to warn shareholders about the current bust cycle. In fact, Wall Street worried that Merck would have a difficult transition during its patent expirations, but Gilmartin was adamant that the company would maintain competitive earnings growth. "He committed to something the market didn't think was possible, and the market was right," says Barbara Ryan, an analyst with Deutsche Bank. "From a shareholder perspective, Gilmartin doesn't seem to acknowledge that as a failure. Therefore, people are skeptical of their abilities going forward." Gilmartin invokes a baseball analogy to explain why Wall Street shouldn't give up on his company. "It's like having a hitter with a great record who's in a slump but who will produce in the future. We just have to keep slugging it out." Says CFO Judy Lewent: "If you strip away the dampening effect of the patent expirations, what you have left is a core business that's doing very well. We have the ability to rebound."

As if Gilmartin didn't have enough to worry about, he is preparing to spin off Medco Health Solutions, Merck's prescription benefits management (PBM) arm. In an IPO scheduled for the last week of June, Merck will put 20% of Medco up for sale. The company plans to distribute the remaining portion to Merck shareholders over the subsequent 12 months.

The divestiture will spell big changes for the drugmaker. Once the separation is complete, Merck will drop from the No. 1 spot in the global pharmaceuticals business to No. 4 (behind Johnson & Johnson, Pfizer, and GlaxoSmithKline) in terms of sales. Merck purchased Medco in 1993 to hedge its bets against what everyone thought would be a more heavily regulated, Clinton-influenced health-care environment. Its competitors followed suit, buying or creating their own benefits management programs. But by the late '90s, Merck was the only major drug firm still in the PBM business. With Medco, the drugmaker thought it had an edge. Merck's smaller-than-average sales force could focus mostly on selling to managed-care operations rather than to doctors, winning millions of customers at a time. The strategy appeared to work. In less than ten years, Medco's revenues grew from $2.2 billion to $26 billion in 2001, accounting for 54% of Merck's overall sales.

But it was hardly a match made in heaven. For one thing, managing prescription benefits is far less lucrative than developing and selling drugs--last year Medco accounted for less than 10% of Merck's profits. Worse, Medco's return on capital was much lower than that of the drug business. On top of that, Medco is saddled with antitrust suits and federal investigations as a result of its ties to Merck. Plaintiffs in the cases allege that Medco steers patients toward more expensive Merck-made drugs instead of cheaper remedies made by competitors. Merck executives say the cases have no merit and deny any wrongdoing. They also claim the legal troubles have little to do with Medco's sale.

In all, Merck executives refuse to call the Merck-Medco pairing a failure. It was, they say, a timely idea whose time has passed. Surely, though, Medco never really delivered on its promise, mostly because the Clinton health plan never materialized. Says CFO Lewent: "We've had a pretty impressive time together, but we really want to go back to focusing exclusively on pharmaceuticals.'' In anticipation of the breakup, Gilmartin is beefing up Merck's sales force, adding 500 salespeople this year.

So it's up to the labs. But as Merck knows all too well, discovering drugs isn't like dreaming up a tasty new soft drink or tweaking next fall's line of SUVs; it's an unpredictable, tedious endeavor. Breakthroughs happen in their own time. As a Merck biochemist puts it, "What we do is look down a lot of blind alleys to make sure they're really blind alleys."

Gilmartin, for his part, is putting his money where his mouth is, infusing Merck's R&D with a $2.9 billion investment, an 18% increase over 2001's budget. The extra funds will go toward things like the anti-obesity drug the company is developing at its San Diego labs. Still in the earliest phases of testing, the drug seeks to tap into the portion of the brain that regulates appetite. "We're looking at solving this problem not through the gut but neurologically," says Peter Kim, Merck's executive vice president of research. Kim lights up when he explains how the drug might work. He's intrigued, he says, by the sheer "elegance" of Merck's sought-after solution. Similar products attempt to attack obesity by fighting fat molecules in the digestive tract, but they come with terrible side effects like intestinal leakage. "Those products are fine--if you like wearing diapers," sniffs Kim.

Merck is also on the hunt for an anti-anxiety drug that doesn't carry side effects like sexual dysfunction and nausea. And the company has entered into scores of joint ventures and licensing deals with biotechnology firms--involving everything from rotavirus vaccines to gene mapping. To make these discoveries and others, argues Kim, "resources and size are not the issues. There are chance events at work here. The degree to which we're successful is dependent on the quality of our researchers. Our strategy can fail only if we have mediocre players."

Rebecca Dias certainly has never been ordinary. Before coming to Merck she published several acclaimed papers on brain activity and learning in rats. And though they will never know it, rat No. 3 and the other denizens of her memory lab could play a small but important role in Merck's future.

FEEDBACK jsimons@fortunemail.com