(FORTUNE Magazine) – There was a time in Jim Collins' life, long before he became obsessed with the stock chart of Walgreen Co., when regular encounters with death were a way of life. Most of them occurred high on the walls of Eldorado Canyon outside Boulder, Colo., where Collins would engage in ritual battles with gravity and fear. Once, as he clung 400 feet up on a vertical crag of granite known to climbers as T-2, his rope unaccountably came untied from his harness, leaving him just a toehold or two away from eternity. He was in the tenth grade at the time.

Twenty-eight years and one mega-bestseller later, Collins is back in Eldorado Canyon, this time grappling with a different sort of vertical limit. Just down the road is his "management lab," where, since his banishment from Stanford University in 1995, Collins and his research team have been tackling one of the biggest questions business has to offer. They want to discover how mediocre companies can become great.

To look at this year's list of America's Most Admired Companies--that is, most of the same royalty that rule the list every year--one might despair that such a leap is possible. Some companies are just born to greatness, it seems, while others have mediocrity thrust upon them. Collins' 1994 book, Built to Last (co-authored with Stanford's Jerry Porras), which plumbed many of these companies for the historical sources of their greatness, seemed to suggest much the same.

Yet therein lay Collins' new quest. Not long after the book's release, he was at dinner with several partners from McKinsey & Co. when one of them leaned over and observed, "We really love Built to Last here, but unfortunately it's useless." Collins asked what he meant. "Well," the partner said, "all the companies in Built to Last were always great. They were never average. But that's most of the world." On a plane later that night, Collins tried to draw up a list. "I started thinking, What's an example of a Westinghouse that became a GE?" he recalls. "What's an example of a Kmart that became a Wal-Mart, a San Francisco Chronicle that became a New York Times, a University of Colorado that became a UC-Berkeley?" By the end of the flight, his list was still blank. Confounded, he printed on a clean sheet of paper, "Can good become great? And if so, how?"

Almost five years later, Collins' quest has finally yielded some answers. They aren't what he, or anyone else, expected.

Ground zero of the quest is Collins' management lab, actually part of an old brick schoolhouse where he attended the first grade. Collins has funded it with half a million dollars of his own money, most of that courtesy of Built to Last: True to its name, the book lasted five years on the business bestseller lists and has now sold almost a million copies.

Inside, a team of part-time researchers--Collins insists on calling them "chimps," after Curious George--meets to feed him a steady diet of data. As his perpetually wide-eyed appearance suggests, Collins' appetite for information is ravenous: He speed-reads books on tape by playing them extra-fast, and he tries to make a full rotation through the magazine rack once a year, dutifully digesting everything from Teen to Modern Bride to Vibe. Laughs Brian Bagley, who serves as Collins' chief chimp: "When we do research, it's with a capital 'R,' baby."

The team began with all the companies that appeared on the FORTUNE 500 in 1965, 1975, 1985, and 1995--1,435 of them. To qualify for the study, a company needed to show a record of sustained mediocrity followed by at least 15 years of outstanding performance. But how to define outstanding? Collins figured that General Electric under Jack Welch provided a pretty good definition: From 1981 to 1995, GE outperformed the overall market by a factor of 2.4 to one. Collins upped the cutoff to an even three. His companies would have to go from average to beating Welch's GE.

If that hurdle sounds high, consider some others. Companies that seemed to owe their shift to some industrywide event were disqualified, eliminating both Coke and Pepsi, among others. Companies that made a leap to excellence, but failed to sustain it, didn't count either. This knocked out the handiwork of both Lee Iacocca (Chrysler) and Stanley Gault, who made Rubbermaid a Most Admired perennial during the late '80s and early '90s. (All qualifiers had continued their success under more than one CEO.)

After nearly a year of winnowing, just 11 companies were left standing. And the winners were...? Nobody you'd much expect. Abbott Labs, Circuit City, Fannie Mae, Gillette, Kimberly-Clark, Kroger, Nucor, Philip Morris, Pitney Bowes, Walgreen, Wells Fargo--a dowdier bunch you'd be hard pressed to assemble. But their stock charts were another matter. "Do you realize that from 1975 to today, Walgreen beat Intel?" says Collins, sounding as if he's giving someone the best news of his life. "It beat Intel nearly two to one, GE almost five to one. It beat 3M, Coke, Boeing, Motorola. And we're talking about a company with more than 50 years of averageness behind it.... Who would have possibly thought?" A composite stock chart of the 11 companies (which Collins is not yet publishing) shows a line lollygagging toward nowhere in particular and then, unexpectedly, and even shockingly, rising into a cliff as sheer as T-2.

In this set of companies, Collins felt he had something remarkable. "We found companies that had been the dregs of utterly oppressive mediocrity that produced those results," he says. "That's what's so exciting. I cannot think of a worse company than Nucor in 1965. Another company had been losing a million dollars a day before its transition.... If there are great CEOs in history, these 11 have to be it." So began phase two of the research: discovering what, exactly, these CEOs did.

Taking a page from Built to Last, Collins first created a control group of 17 "comparison companies" that shared all of the 11 companies' key traits save one: success. Then Collins read--and his team meticulously coded--every newspaper and magazine article written about each of the 28 companies since their inception, 5,979 in all. The researchers interviewed every living member of the executive teams responsible for the shifts, producing 3,000 pages of transcripts. Then came the serious data crunching. One unlucky chimp spent the summer searching for a correlation between executive compensation and performance, only to find none. Ditto for a company's organizational structure.

So how did these companies escape mediocrity? Instead of answering the question outright, Collins makes his interlocutor hazard a guess. This one guessed what most people do: that some charismatic outsider must have come in and shaken things up. At which point Collins paused for effect and replied that, in fact, ten of the 11 CEOs were insiders--and largely anonymous insiders, at that. Anyone heard of Lyle Everingham, Darwin Smith, Colman Mockler, Fred Allen, or George Cain? So much for the savior CEO hypothesis.

Surely, then, the answer must lie in strategy. These folks saved their companies by plotting some bold new course, right? Wrong again. "I'm not trying to debunk or to build up strategy," says Collins, almost apologetically. "It's just that the data clearly show that these companies didn't start there--and the comparison companies did," with often disastrous results.

This willingness to follow his data straight off a cliff, if that's where the data lead him, is partly a function of Collins' unique status: He belongs to no church of management thought--no strategy-centric consulting firm, no leadership-oriented management department--where his unorthodox conclusions might be construed as heresies. (He's a self-described "strategy agnostic.") Yet he didn't end up here voluntarily.

Collins began at Stanford Business School in 1988 as a lecturer in entrepreneurship, following stints at Hewlett-Packard and McKinsey. His accessible, high-octane teaching style (he was known for wearing ties that related to the day's case study) earned him throngs of adoring students. It also set him at odds with the rigidly academic environment of Stanford, where writing for something as popular as Harvard Business Review could be a career-limiting move. And by all accounts, including his own, Collins worsened the friction by letting his opinions rip, casually disparaging other Stanford departments in his lectures.

Meanwhile, his popularity was becoming something of a problem. Each student at the business school was given a so-called silver bullet that could be "fired" just once to get a first-choice elective. The fact that more than half the student body was using it for Collins' course inflamed jealousies, all the more because Collins lacked a Ph.D. and thus, by academia's Procrustean standards, legitimacy. The astonishing commercial success of Built to Last only served to confirm his status as persona non grata.

Collins didn't see what was coming until it was too late. The faculty enacted a policy requiring lecturers to have qualifications he pointedly lacked--his supporters trenchantly called it the "Collins rule"--and effectively showed him the door. "His sin," argues one of his faculty allies, "was being too popular." Collins says simply, "I was incredibly politically inept."

Being ejected from the school where he had earned both his undergraduate and MBA degrees plunged Collins into personal crisis. But it also presented an opportunity. "When I had taken career tests years before that, they always came up with two contradictory things: You should be either an academic or an entrepreneur," he says. "So I merged the two and became a professor of entrepreneurship. But finally I realized there was something kind of hypocritical about that. Here I was, trying to get my students to carve their own path and to do something entrepreneurial, and here I am at Stanford University.... I remember looking at my academic colleagues and saying, 'Man, there's no risk in this.'... So I decided to turn it inside out, to reverse the words, and become an entrepreneurial professor."

Collins and his wife, Joanne Ernst, a high school track coach and former Ironman triathlon champion, picked Boulder as their new base, mostly for the climbing nearby. And here, free of academia's constraints, Collins invented a new role for himself: that of the pure management scientist, captive to no orthodoxy or set of assumptions--unroped, as it were, to clamber about his mountains of data in search of what he calls "truth with a small 't.'"

At the same time, Collins didn't want to fall into garden- variety gurudom--the breezy assertions, the books-as-marketing-vehicles--and set up a series of mechanisms to ensure he wouldn't. There would be no Built to Last field book or video series, no Jim Collins Inc. to capitalize on his fame. His consulting activities would be limited to one day per year per company, and then only if the CEO flew to Boulder. A three-person "council" of himself, Joanne, and Brian Bagley (not to be confused with Collins' "personal board of directors," consisting of friends and mentors) would have veto power over any career-related engagement. And perhaps rarest of all among management-guru types, he resolved to apply his teachings to himself, actively purging his life of contradictions. Because he taught that jobs were an advanced version of indentured servitude ("You used to own people by owning their land; you own them now by owning their time"), he insisted on hiring his researchers only on a contract basis.

For all his attempts at scientific purity, though, all his self-styled agnosticism, Collins did retain one prejudice: an abiding distrust for the concept of leadership. "I've never believed in leadership," he says. "In the 1500s, people ascribed all events they didn't understand to God. Why did the crops fail? God. Why did someone die? God. Now our all-purpose explanation is leadership.... We have basically lots of witchcraft, lots of religion, and very little understanding."

His earlier research had left Collins with the Tolstoyesque view that corporate leaders were, in fact, the slaves of much larger organizational forces (or, as the master himself put it, like children holding a pair of ribbons inside a coach and imagining they're driving the horses). It wasn't the leader's personality that mattered so much as the organization's personality, and Built to Last's almost radical achievement was to make those personalities seem as palpable and idiosyncratic as human ones. In a sweeping approach that owed as much to business historians like Alfred Chandler as it did to conventional management thought, the book showed how such companies as 3M achieved greatness not because of one great disciplinarian but through a larger "culture of discipline" erected and reinforced over time. It convincingly portrayed Jack Welch as the product, rather than the producer, of GE's success.

The authors even went so far as to classify charisma as a liability. For instance, Rubbermaid's decline (and eventual sale) after the departure of Stanley Gault was widely interpreted as proof of Gault's executive genius. But Collins interpreted it as just the opposite: By organizing Rubbermaid around his outsized personality, Gault had failed to build the mechanisms that would allow the company to excel without him. Wal-Mart's Sam Walton, by contrast, successfully "overcame" his native charisma to do just that. "You can't really assess a CEO's performance until about ten years after that CEO is gone," said Collins. His voice growing resonant with frustration, he added, "We attribute IBM's stellar returns in the 1990s to Lou Gerstner, but we're confusing correlations and causes. Could Gerstner have done to Burroughs what he did to IBM? No, no, no!" There were three loud thwacks as he struck the table in his management lab.

Collins traces our current fixation on hero CEOs to the publication of Lee Iacocca's book Iacocca. "Starting in about 1985, there was an obvious and significant shift in the way [the media] covered business," he observes, having read a century's worth of business journalism. "Instead of covering companies, it began covering people." That coverage, Collins fears, has begun to influence managerial practice itself. "I believe that the celebritization of CEOs, were it to continue, would be one of the most damaging things that could happen to the long-term health of companies," he says. "It's a reflection of a great cultural yearning for new gods so that we don't have to think. If we don't think, we won't understand. We don't understand, we will lurch."

Which brings us back to the question: How did the 11 mediocre companies become great? By way of explanation, Collins relates the story of Darwin Smith, who took over Kimberly-Clark in 1971. A frumpy man who shunned the spotlight and drove tractors for fun, Smith inherited a company whose industrial paper mills were paragons of mediocrity. But he did not enter the job touting any grand strategic solution. "When asked what was their vision and direction for the company, every single one of [the 11 CEOs] said, 'I don't know,' " says Collins. "They were more like Socrates than Caesar: They led with questions, not answers."

The questions Darwin Smith asked aimed one level deeper than strategy. What, he asked, could Kimberly-Clark be best in the world at? What would ignite its passions? What would best drive its economic engine? For two years he repeated his questions quietly but insistently, sparking internal argument, searching for a penetrating understanding of the organization's future.

It wasn't the sort of take-charge performance that soothes boards of directors. But those who read Smith's deliberateness as lack of will were mistaken. After two years of discussion, Smith settled on a radical course: Sell the mills. Kimberly-Clark would unload its core business entirely--even the company's mill in Kimberly, Wis.--and plow the proceeds into its consumer-products business, which until then had consisted mostly of Kleenex. Twenty-five years later, Kimberly-Clark was beating Procter & Gamble in six of eight paper-product categories.

Yet here was the strange thing. When Collins' team asked various executives to pinpoint the key moment their companies made the shift, most of them couldn't do it, even in such a seemingly dramatic instance as Kimberly-Clark's. "There was no one magical event, no one turning point," one executive told the researchers. "It was a combination of things, more of an evolution, though the end results were dramatic." Said another: "It wasn't a single switch. There was no single epiphany moment or bright light that came on. Little by little, things became clearer and stronger."

Collins explains the phenomenon thus: "I'd like you to think of a big, heavy flywheel. A big metal disk that's just massive. And your task is to get that flywheel turning as fast as possible. And so you start pushing on the flywheel, and you're pushing and you're pushing really, really hard. And the flywheel makes one giant, slow, creaky turn, and you're tired. But you keep pushing in a consistent direction. And the flywheel starts to pick up a little more momentum, and now it's gone two revolutions, and it's got a little more speed behind it. And then three, and then four, and then five. And then at some point, there's this sense of breakthrough, where all of a sudden all that weight that was against you is sort of with you. You're still pushing hard, but it's going 100 or 200 or 300 rpms. Whoosh! And now I'm going to ask you, 'Well, what was the one big push that made it go?' "

The comparison companies, by contrast, did try to jump straight to breakthrough with one big push. They hired high-profile CEOs from the outside. They unveiled the CEO's bold new strategy. They made audacious acquisitions. They launched change initiatives and held pep rallies, while the media often obliged with flattering stories. "The comparison companies often acted with bravado," says Collins, "while the good-to-great companies acted with understanding." He calls the resulting dynamic a "doom loop": bold but undisciplined steps, leading to disappointing results, leading to more bold but undisciplined steps. The recent travails of AT&T come to mind. "It is an absolute textbook case of the doom loop," Collins opines.

To be clear: Many of the upwardly mobile companies also adopted new strategies and made big acquisitions. During the 1960s, Nucor shifted from making nuclear-power equipment to operating steel mini-mills. Walgreen bailed out of the food-service business to focus on becoming the leader in convenient drugstores. Kroger decided to replace its old-style grocery outlets with superstores, investing heavily in scanning technology in the process. The difference, Collins emphasizes, is that none of them began with those steps. When they did make acquisitions, it was to accelerate a flywheel that was already moving.

Nor did they react overmuch to external events. During the Internet scare of 1998 and 1999, when slogans of CHANGE OR DIE! were all but graffitied on the subway, Walgreen obstinately stuck to its corporate credo of "Crawl, walk, run." Its refusal to act until it thoroughly understood the implications of e-commerce was deeply unfashionable, but as Collins notes, Walgreen is the epitome of the inner-directed company. "One of the defining elements of mediocre companies is, they're fundamentally motivated out of a fear of being left behind," he says. "It's the difference between reactionary lurching and cumulative building."

In the end, Collins was wrong about the leadership thing. Faced with the data, and some heavy pushback from the chimps, he conceded that leadership seemed to matter after all, if not in the usual sense of the word. He's now convinced that the quiet, almost willfully uncharismatic style of people like Darwin Smith--their humility before organizational forces larger than themselves--played a key role.

"Did you ever read the book Undaunted Courage?" Collins asks, referring to Stephen Ambrose's history of Lewis and Clark. "At the end of the book, Meriwether Lewis commits suicide. Clearly the happiest time of his life was when he was in the boat going west but didn't know what was going to be there when he got there. The sense of anticipation was truly a blissful time for him. I love being in the boat and not knowing what's on the West Coast, and we're going to find out what's there." Now that the journey is almost over--his book Good to Great is scheduled to appear in the fall--one wonders where Collins will direct all that obsessive energy.

In a Boulder coffee shop after an afternoon of hiking in Eldorado Canyon, he noticed something through the window. "Look," he said, pointing to a silhouetted mountain ridge. "Good to great."

Come again? Then I saw it. The ridgeline continued horizontally from left to right, like the stock chart of a mediocre company, before starting a steep, jagged rise. Good to great. Collins grinned and walked outside.

FEEDBACK: juseem@fortunemail.com