(FORTUNE Magazine) – Next time an invisible hand yanks you off an overbooked flight and shreds your travel schedule, and you find yourself wondering whether the airline industry will ever grow up and behave like an adult member of the infrastructure, it may help to bear in mind the following simple rules:

(1) To understand what keeps airlines aloft, it is instructive to think about sex.

(2) To understand why air travel is so frequently wretched, it is instructive to think about bananas.

Let us explain.

The airline business, like the movie business and professional baseball, is so fundamentally sexy that many people can't resist it. Ever since Eddie Rickenbacker bought Eastern Airlines and Howard Hughes bought TWA, investors have been drawn to airlines for reasons going far beyond money. For the pilots, mechanics, and flight attendants who actually spend their days in and around planes, the attraction is, if anything, stronger ("jet fuel in our veins" is how some describe it). Passengers aren't immune to the industry's sex appeal, either: Flying often involves an implicit (if tiny) risk of death, the close proximity of strangers, and liquor.

For all its ineffable allure, though, the airline industry is like the banana business. Airplane seats are a commodity--despite heroic attempts at product differentiation, passengers think one airline's seat is pretty much like another's-and when a plane takes off with empty seats, the commodity is spoiled. It is better by far, from an airline's point of view, to lure passengers from competitors at deep discounts than not to sell seats at all. "We sell a perishable product, and it's like the last banana in the store: You can either get a penny for it or you can get nothing for it, so you sell it for a penny," says John Dasburg, chief executive of Northwest Airlines.

Combine these characteristics and what do you get? Vicious, endless fare wars that in most businesses would quickly thin out the players but that in airlines--the commodity with sex appeal--have helped produce a record of chaos and wreckage rare in any industry and almost incredible in one so important to the nation's and world's economy.

You get more: discontent ranging toward misery for nearly everyone involved. Investors just keep getting hammered. In the past five years the commercial aviation industry has lost pretty nearly all the money it made since the first passengers flew commercially in 1914 (St. Petersburg to Tampa, $5 one way). Since airlines began competing freely on routes and fares 17 years ago, 120 have gone bankrupt, some more than once. The Standard & Poor's airline index fell 30% last year. Warren Buffett's Berkshire Hathaway bought $358 million of convertible preferred stock in USAir in 1989, and Buffett rues the day. He recently told students at the University of North Carolina's Kenan-Flagler Business School that when the Wright brothers flew at Kitty Hawk, "if there had been a capitalist down there, the guy should have shot down Wilbur."

This business also punishes its employees. More than 100,000 have been laid off since 1989. Many others have taken pay cuts. You could blame the top executives, but they're not exactly above the carnage: The CEOs of Continental and Kiwi International have lost their jobs in the past six months.

As for the passengers--well, you probably know. Yes, average ticket prices have declined steadily since deregulation in 1978, and millions of people have gone flying who otherwise wouldn't have been able to afford it. Yet today's fliers, especially business fliers, don't seem happy, and it's hard to blame them (see following article--or ask the person in the seat next to you).

That's a lot of suffering that no one wants, but at least people in and around the industry long thought they understood its causes. Now many aren't so sure. A small but growing group of industry experts believe the conventional explanation of chronic airline turmoil is wrong--and that the real reasons suggest more pain for travelers and investors and maybe, eventually, calamity for the industry.

The commonly accepted view goes this way: Pricing madness and mammoth losses have resulted from bad policy, bad management, and bad luck--problems that have masked progress toward an eventual, conventional settling down of the industry. The past five years have seen an exceptionally rotten string of misfortune: Misguided expansion plans led to overcapacity, Iraq's invasion of Kuwait caused fuel prices to double, and the subsequent war discouraged air travel. Then the recession kicked the crutches out from under a limping industry. Only now is the business approaching some kind of stable equilibrium. As Herb Kelleher, chief executive of Southwest Airlines, puts it, "Deregulation is just now reaching its denouement."

By contrast, the new view says the trouble isn't bad luck; it's the very nature of the business--and that means the trouble isn't going away. Get used to it. The airlines, say backers of this view, are a highly visible example of a competitive market that is, essentially, absurd. Loosely associated with Lester Telser, a prominent economist at the University of Chicago, they argue that the airline industry is an economic machine programmed for self-destruction or something close to it.

The argument, at its simplest, goes like this: No matter how many employees are laid off or labor concessions gained, the fixed costs of aviation--planes, fuel, facilities-are destined to remain relatively high; it is simply more expensive to fly through the air than, say, to grow bananas. The marginal costs of adding passengers on a partly filled flight are negligible--a can of soda, some pretzels, a little fuel, and a few dollars to process the ticket. So you needn't charge much to make that perishable seat worth selling. Result: Last year 92% of airline passengers bought their tickets at a discount, paying on average just 35% of the full fare. In an industry with high fixed costs and low marginal costs, competition may produce a market that never settles down. Economists say such a market has an "empty core."

"Fixed costs of an airline depend on the distance the plane goes, or whether it takes off or not, but they don't vary based on the number of passengers," Telser says. "What'll happen is that, depending on demand conditions, it may turn out there is some carrier that can make a deal with a subset of customers, but on terms that will screw things up for the industry as a whole."

Imagine, for instance, a market in which a taxi holds two people, and only two. Three people are waiting at a taxi stand, bound for the same destination, and two taxis show up. How much it costs a taxi to make the trip doesn't depend much on the number of passengers. One taxi driver offers to carry two passengers for a total of $20. The second driver can try to make the same amount of revenue by offering the third passenger a fare of $20, but that passenger will likely take a bus, or not travel at all, rather than pay that much. So the second driver tries to upset the first driver's agreement, undercutting his fare for two passengers. You can see what happens: Any price agreement struck by a coalition of two passengers and one taxi can be upset by a slightly better offer from the other taxi (or the other passenger), cascading until it is no longer profitable to operate one of the taxis.

It may seem too simple a puzzle to produce chaos, but from it unspools a fiercely complex branch of economics called core theory. Does it really describe the airline industry? "It may be so," says Alfred E. Kahn, an architect of the airline deregulation act and often called the father of deregulation. "I don't have any conclusive proof. In contrast to the people from the University of Chicago, I don't have such firm convictions." He prefers to believe that what he calls "this embarrassingly intense competition" by the airlines can be moderated if costs are brought into line.

Core theory "really amounts to saying that competition just isn't possible in some industries, and that kind of proposition is bound to generate controversy," says William Sjostrom, senior lecturer at University College Cork in Cork, Ireland. Sjostrom compares the problems of airlines with those of shipping lines, which have similar cost structures. In a study of container shipping in the Pacific, he found that this market also has an empty core and that this led to the development of shipping conferences--legalized tariff agreements between shippers and customers.

Of course, in the U.S. air transport market, price coordination is illegal and reregulation unthinkable. So what does this view of the industry mean for air travel? "Eventually what happens is, the situation gets so bad that people realize that some very drastic reforms are necessary," Telser says. "The railroads are now at a point where they are becoming healthy. They all became bankrupt, and they all had to work out solutions. I think we may be approaching that point in the airlines." Not all that adventurous a prediction: Two of today's major airlines, TWA and Continental, were recently in Chapter 11, and at least one other could soon pay a visit.

This view suggests that the industry's violent cost cutting amounts to a gigantic fool's errand. The idea horrifies airline executives, who agree with Kahn that the industry is moving rationally toward a new, stable model, built on the failed tries of the past. Recall that immediately after deregulation the early betting was on new, low-cost carriers, exemplified by People Express, which could fly without established airlines' high costs for labor and administration. Big airlines responded by developing hub-and-spoke systems to scoop up connecting passengers and by introducing frequent-flier programs to build customer loyalty. Some big airlines indeed collapsed under their own weight--Braniff, Eastern, and the flag carrier, Pan Am--but those that survived, to the surprise of free-marketers, ate the low-cost upstarts for breakfast.

During the mergers and bankruptcies of the mid-1980s, a new model of success emerged: Oligopoly would rule. A few bloodied survivors, it was thought, would divide up the market the way the Big Three automakers had. Major airlines expanded their hub-and-spoke systems and ordered huge numbers of planes. "Back then there were a lot of magic words, like critical mass," Kelleher recalls. "I said, you know, the people who talk about critical mass are obviously not nuclear physicists, because when you reach critical mass, that's when you blow up."

Kelleher didn't build hubs. He flew point-to-point in short-haul markets, charged fares that were well below competitors', did away with assigned seating, and used only one kind of plane, the Boeing 737. A genuinely charismatic, charmingly profane man, he won astonishing productivity from Southwest employees. When the airline business went through its calvary, losing about $12 billion between 1989 and 1993, Southwest alone showed a consistent profit.

Wall Street swooned, and a new consensus seized the business: Hubs were expensive assets, and survival would go to the cheapest. A new generation of low-cost, low-frills carriers has taken off and expanded at a remarkable rate. Low-fare service had spread to 47% of the top 1,000 city-pair markets under 1,000 miles apart as of last December, up from 21% just 18 months earlier, says a study by the Airfare Management Unit of American Express Travel Related Services Co. (see map). ValuJet, Reno Air, Kiwi-altogether 69 carriers have been launched in the past three years, and ten more await certification from the Federal Aviation Administration. Some of the new airlines are charging fares so low, they cause problems for bus companies and Amtrak. Flying mostly short-haul routes in high-density markets, as Southwest does, they move in like hermit crabs to occupy the gates and ticket counters of big airlines for a few flights a day, often slapping up temporary signs. Some don't pay to have themselves listed in computerized reservation systems, and others don't issue paper tickets at all, simply taking reservations by phone and issuing boarding passes on a first-come, first-served basis.

Low-cost airlines generated about $1.4 billion in revenue in 1994, up from $450 million in 1992 and next to nothing in 1989, says Paul Karos, an airline analyst at CS First Boston. Robert W. Harrell, who heads Amex's Airfare Management Unit, says the penetration of low fares in the nation's top 1,000 city-pair markets has leveled off in the past nine months; he expects it to decline somewhat. "The number could reach some equilibrium point, though I'm not sure anyone knows what it really is," he says. "Maybe it's about 40%."

Whatever it is, it's high enough to scare the major airlines. They have responded to the upstarts as aggressively as might be expected, sometimes matching or even undercutting their fares on overlapping routes. At least three low-cost carriers say they have taken complaints of predatory pricing to the Transportation Department.

More important, the majors have seen the lesson of low costs and have responded with vast, painful cost-control programs. United last summer completed an employee buyout of 55% of the company in exchange for $4.9 billion in labor concessions. Delta has laid off 10,000 employees and outsourced many of its "below the wing" (noncustomer service) operations. American is trying to shave $1 billion from its costs by negotiating productivity improvements with its unions, laying off managers, grounding jets, and withdrawing jet service from 29 cities. USAir in March announced it would operate 170 fewer flights. USAir has closed its hub at Dayton, as has American at San Jose and Raleigh/Durham, and Continental at Denver. Delta said it would cap commissions to travel agents, and its biggest competitors followed. When you're squeezing costs, you don't buy many new planes: More than a quarter of the commercial airplanes in service in the U.S. are now over 20 years old; this country's fleet is among the oldest in the developed world.

Two major carriers even joined Kiwi, Reno, and the rest at the startup party. United Airlines has established a low-cost "airline within an airline" it calls Shuttle by United to take on Southwest directly. Continental tried something similar last year, expanded too quickly, and ended up losing so much money that its chief executive was ousted. "We set out to prove that you could outrun your headlights in this business, and we ended up in a ditch," says the new chief executive, Gordon Bethune.

Today the betting--at least among believers in rational markets--is on the venerable strategy of product segmentation. Some airline executives point to an emerging understanding that different classes of airlines can serve different kinds of passengers without killing each other. Northwest CEO John Dasburg says the new low-cost, short-haul carriers "are viable products, and they will endure, for the same reason Motel 6 endures. This product will be defined. It will be located in the top hundred markets that are 750 nautical miles or less from each other."

That's attractive to passengers traveling between those city-pairs who don't need to connect to other flights. But another market segment is fliers with complicated itineraries. For them there is simply no more efficient alternative than a true, full-service network operating through a hub-and-spoke system. As Thomas J. Roeck, Delta's chief financial officer, puts it, "Nobody can get you from West Palm Beach to Boise the way we can."

United is trying to serve both markets separately. Chief Executive Gerald Greenwald is increasing the number of flights operated by the low-frills Shuttle by United, which started service last October, competing in Southwest's West Coast territory. Greenwald says: "I think there's room for both of us. Prices are stable to gently up." At the same time, using some of the savings from the carrier's employee buyout last summer, he is investing in customer service on longer routes: adding amenities, improving meals, and increasing the number of flight attendants. He foresees a time when full-service air travel will feature in-flight video shopping services at each seat and perhaps, on international routes, gambling. "I fundamentally believe that there are many markets in the airline industry," Greenwald says, "and what's exciting is that the same passenger may be in different markets on different days."

This year is actually supposed to be a relatively good one for the airlines overall, with fuel prices flat and the economy still robust after several years of recovery. Some airline analysts predict the industry will even post a slim profit, after breaking even last year and losing $1.1 billion the year before--though airlines are issuing dire warnings about the possible effects of a fuel tax scheduled to take effect in October, which an industry spokeman says could cost the airlines $500 million. Could this be a harbinger of equilibrium, promising calmer journeys for both the airlines and the passengers they serve?

"The economic historian in me would suggest yes. However, the airline executive in me can't see it," says Michael J. Durham, chief financial officer of American Airlines. Market segmentation may sound promising, but maybe competitors will just pound each other to death within the segments. Durham has seen promising developments before. During an especially destructive price war in 1992, American Chief Executive Robert Crandall tried to lead the industry toward a sensible pricing structure, eliminating discounts and establishing four classes of simplified fares. That lasted a matter of months; rather than following American's lead, competitors fell over one another to undercut its prices. "We tried to create an economic pricing structure that consumers would like and that would cover our costs and make a reasonable profit for our shareholders," Durham says. "It failed. I don't think there's much appetite for trying it again."

As Rich Golaszewski, an executive vice president at transportation consultants Gellman Research Associates, puts it: "Crandall tried to lead the way out, and it fell apart because on day two, everybody said, 'Good, Bob raised his prices; let's steal all his passengers.' That is the systemic problem in the industry: It may be pricing-unstable. It's core theory, and the core is empty."

As the industry evolves, that possibility remains. But so does another key trait, flying's enduring sex appeal. Consider: Kiwi, started by grounded Eastern and Pan Am employees and proudly named for the flightless bird, is funded largely by its workers. Pilots pay in $50,000 apiece to get jobs; other employees, $5,000. People can't stay away from the airline business. Reflects Alfred Kahn, the economist who helped deregulate the airlines: "There does seem to be something about airplanes that drives otherwise rational investors out of their minds."

So even if the market has an inescapable imbalance, it may well stumble along indefinitely, propelled by an urge more powerful than return on investment. That will mean more bankruptcies, more new entrants, more ferocious cost cutting, high-risk-uncertain-reward investments, rock-bottom fares as far as the eye can see, which will buy service that people will grouse about, nice service if you want to pay for it--and no matter what happens with the economy or fuel taxes, one of the world's toughest ways to make a dollar will just stay tough.

Oh, and by the way: Governments of other countries, impressed by the cost savings deregulation has conferred on travelers in the U.S., are determined to follow the American example. Canada and the U.S. in February reached an "open skies" agreement permitting each nation's airlines to compete in the other's markets. The European Union is set to deregulate air travel by April 1, 1997. April Fool's day.

For passenger angst, see next story.