(FORTUNE Magazine) – With their black jackets whipping in the wind, bleary-eyed traders and brokers stamp out their cigarettes before lining up at the doors to the New York Stock Exchange. Eager to hit the floor before the opening bell, few pause to admire the white classical architecture that separates the structure from its neighboring buildings. The six Corinthian columns, currently draped with the American flag, are impressive enough. But sweeping into the sky is a pediment completed in 1903 by sculptor John Quincy Adams Ward. It depicts a woman in a flowing robe, her arms stretched out in front of her. Alongside her are a handful of figures laboring with obvious strain. Ward's title for his art? "Integrity Protecting the Works of Man."

Well, it now seems that Lady Integrity wasn't spending enough time inside the building. As a result, the Works of Man are facing an onslaught of regulators, investigators, lawyers, angry institutional investors, and increasingly frustrated NYSE members. Rival exchanges are pouring on the heat, the New York Attorney General has filed suit, and even the former CEO, Dick Grasso--whose stunning $188 million pay package led to much of the current brouhaha--is countersuing his old employer and its new chairman, John Reed. The salacious trash-talking on all sides has been grist for newspaper headlines for the past 18 months, but missed in the fur-flying and fury is that something momentous is actually happening to the once-impenetrable NYSE.

The 211-year-old symbol of American capitalism and commerce is about to change in a big way. (Really.) Investors large and small are likely to benefit when all is done. But the biggest surprise may be the guy who's going to put this transformation in place--a quiet, earnest, 48-year-old technocrat from Goldman Sachs named John Thain.

Tapped by Reed to be the NYSE's new CEO last December, Thain has been confidently stepping into the giant footprint--and even bigger hole--left by Grasso. The two men couldn't seem more different on the surface. Grasso grew up in Queens, dropped out of a small, local college, and clawed his way up the ranks of the exchange to the sixth-floor suite. He was a guy famous for both bullying and rallying the troops, and is still beloved by many. Thain is from Antioch (Illinois, that is). A small-town boy, who despite MIT, Harvard Business School, and 24 years at Goldman, well, still sounds like a small-town boy.

Thain's task now is to avoid the sideshow debate over his predecessor's legacy and the continuing litigation over Grasso's compensation--and focus instead on the meatier issue. And that's the role of the NYSE's in-house floor traders, known as "specialists," a clubby and insular group who control much of the traffic of shares through the exchange. The debate over the specialists reached a fever pitch in March, when NYSE and SEC investigators found that these traders had bought and sold stocks in front of their customers' orders. The "improper proprietary trading" involved 2.2 billion shares over a three-year period and cost customers $155 million, regulators found. The specialists agreed to pay $242 million in restitution and fines--one of the largest in the exchange's history--to settle the charges.

While the sins were modest compared with the overall number of shares traded on the exchange (equal to less than two days' trading volume), they confirmed what many investors have long believed--that the Big Board had morphed into a casino where the house always wins. The revelations have even brought institutions that normally shun public spectacle, such as Fidelity Investments--which alone accounts for up to 5% of the NYSE's total trading volume--to come out swinging against the exchange.

But even without the loud complaints from important customers, structural change at the NYSE may have been inevitable. Blame the same market blahs that have haunted ordinary investors. NYSE trading volume--how floor members make money--has been flat over the past three years. In the late 1990s, by comparison, volume had been averaging 25% growth a year. And institutional investors are using the soft market to drive down commission costs. "I had a client call the other day asking for a 30% cut in his commissions," says a floor broker who didn't want his name used because he feared other customers would ask for the same deal. "He's only paying 1w penny a trade. I told him, 'Hey, I'm trying to make a living here.'"

The specialists have also seen their businesses upended by the move from trading in fractions to trading in decimals. Before 2001, stocks traded in fractions of eighths and sixteenths (12.5 cents or 6.25 cents, respectively). A specialist who bought a stock at 108 and sold it to someone else for 104 could pocket the 12.5-cent difference. Today that spread--the price gap between the bid (buy) price for a share and the ask (selling offer)--has narrowed to a penny. The market capitalization of publicly traded LaBranche & Co., the largest specialist firm, has been halved, to $474 million, in the past year. "The specialists have been hit by the perfect storm in the last three years," marvels Rick Ketchum, the NYSE's new head of regulation. "Decimalization reduced spreads, trading volumes bottomed out, and regulatory costs--from fines to investments in technology--have increased to ensure it doesn't happen again."

Making sure it doesn't happen again, though, isn't as easy as it sounds. For it cuts to the very nature of the exchange. In their role of matching buyers and sellers, floor brokers and specialists routinely put themselves in the middle of trades. Brokers are paid commissions for the service. Specialists, however, make their money by capturing the spreads. They do so with information--called "color" in trader parlance.

Here's how it works. A broker who walks up to the specialist booth for AT&T's stock may learn that earlier in the day Bank of America and Charles Schwab were big sellers, but that Lehman Brothers and Bear Stearns now have buyers coming in at those price levels. The broker then scurries back to his booth to offer that color to investors, who may then trade on it. When played by the rules, the game isn't illegal, say NYSE officials, because the brokers and specialists are only passing along either historical information about transactions or indications they've received of future interest in a stock. As the price of a stock moves, the trades may or may not occur. What further complicates matters is that a specialist is also allowed to buy or sell shares in the stock he's overseeing--though he does so only about 10% of the time. And here's where that color gives him an edge over everyone else in the market.

Needless to say, institutional investors have a love-hate relationship with these middlemen. When they want information or need someone to buy their stake, they're happy to use the brokers and specialists. But they can't ignore that nagging voice in their head that whispers they're the suckers in this game. In a poll taken earlier this year, nearly three-quarters of institutional investors surveyed said their biggest challenge was dealing with the inside dealers on the floor of the exchange. Among the gripes that are appropriate to print, most got straight to the point: "The specialist will rip you off."

The NYSE floor traders, naturally, defend their role as needed traffic cops at an extraordinarily busy intersection. Because they are encouraged to step in and buy even when the price is falling and no other buyers emerge, they can also lose money on trades. They therefore provide essential liquidity to stocks in free fall or to those not-so-blue-chip stocks that are more thinly traded than, say, GE or IBM.

Or so the story goes. Nasdaq, which performs all its transactions electronically out of an eerily quiet, fully automated computer center in Trumbull, Conn. (you can ignore that flashy storefront in Times Square), has no paper-strewn floor, no specialists facilitating trades. Yet that market handles daily volume of 1.8 billion shares, compared with about 1.4 billion on the NYSE.

Pushing automation at the NYSE (which executes about 8% of its trades electronically as it is) would in essence remove the ability of floor traders to squeeze between the buyer and seller and grab the spread for themselves. But, then, that would take away the good human element from the process --the element that keeps the market orderly and ensures liquidity. Advocates of the current system say it is human beings that have established the NYSE's reputation as the premier global marketplace. More than $9 trillion in stocks trade on its floor each year, after all. And companies valued at $17 trillion pay hefty annual fees simply for the privilege of being listed on its storied board. Defenders say that it was that human element that got the market up and running quickly after 9/11, showing the world that America's central role in commerce couldn't be crushed. Taking people out of the equation now would make the exchange unrecognizable.

Oh, and one more thing. The people who earn their living on the floor--the ones who are likely to lose the most in any reform--also own or lease the 1,366 membership seats of the exchange. Ironically, it's up to them to approve any changes to the NYSE's structure. "The brokers and specialists on the floor are like a union," says a head equity trader at a large Wall Street firm. "Thain has to get the union to accept concessions that may hurt some of them so that the whole exchange doesn't fall apart."

It's a job that, if you do it well, nobody's completely happy. And Thain is now running not only against history but against the clock too. The SEC, under William Donaldson, is in the process of changing the nation's trading rules, which will force the exchange to go head-to-head with electronic communications networks (ECNs) such as Nasdaq's Market Center, Instinet, and Archipelago. The new regulations are likely to say that if the NYSE cannot match its rivals' automatic-execution platforms, investors can take their trades elsewhere. Right now--surprise--they typically can't.

That's because of a decades-old piece of legislation known as the trade-through rule. It states that brokers must execute a trade for a listed stock on the market that offers the best price. The regulation was designed to protect small investors from having their trades ignored. In practice, though, critics say the rule gives the exchange an unfair, monopolistic advantage.

NYSE members brag that the exchange posts the best price 94% of the time and that that's why it gets the overwhelming share of the trading volume. But institutional investors grouse that it can take up to 30 seconds for a broker to negotiate a share price with a specialist--a virtual eternity in today's market. (The NYSE counters that it's more like 14 seconds.) By then the price may have moved far away from the point at which the investor wanted to buy. Typically the buyer has to grin and bear it. "The specialists hold the order up, asking if anyone wants to improve the price," explains Gus Sauter, chief investment officer at Vanguard Investments. "By the time anyone on the floor has decided whether he wants to participate, 15 seconds may have passed, and the market could be in a very different place."

The SEC proposed changes to the trade-through rule this spring. The reform that appears to have the most momentum would allow investors to ignore the best price for a stock on the NYSE (or other nonautomated market) and opt for speedy execution instead, as long as the price difference was within a few cents.

To compete under the new rules, the NYSE needs to become much more like its ECN rivals--which would entail a true metamorphosis. It does have an electronic trading platform in place, called Direct+, which executes trades without any human involvement. But it's pretty much useless to most institutional investors. That's because, among other problems, Direct+ handles only orders of 1,099 shares or fewer. And the trades must be limit orders--they have to be executed at specific price points (again, forcing investors to signal their intentions to the humans on the floor). And finally, investors can enter only one order for the same stock every 30 seconds. Thain is going to lift all of those restrictions.

To keep the brokers and specialists from outright rebellion, Thain is trying to give them the ability to pause an electronic transaction in a particular stock if trading becomes extremely volatile (e.g., after an earnings warnings or merger announcement). But the details have yet to be worked out.

It is a doozy of a high-wire act. If mutual funds and pension funds prefer the NYSE's improved electronic platform, they are likely to trade even more shares on the exchange. But even with an uptick in volume, brokers and specialists will get less business--though Thain contends that floor traders will still make money. "I think you'll see the large, very liquid stocks and the low-dollar-priced stocks trade electronically," he says. "But we have 2,700 listed stocks. The vast majority will still trade with specialists and brokers." Others predict that as much as half of the current volume moving over the NYSE floor will be done by a computer. The actual number could determine the fate of many on the floor. "This change is going to be hard on everyone," says Andy Brooks, head of equity trading at T. Rowe Price. "I hope the exchange can find a happy medium."

Some NYSE floor folk have seen the writing on the wall. JNK Securities began beefing up its own technological capabilities 18 months ago to offer its institutional clients faster, electronic execution. "We're trying to give our clients our version of the Green, Gold, and Platinum Amex card. You want to do electronic trading, you won't pay much. You want to talk to our guy on the floor who's been there 25 years, you'll pay slightly more," says managing director Orest Kyzyk. "We had to either build a defensive position or wait and die."

Investors are likely to benefit whether floor traders survive or not. Increased competition and new technology has to drive trading costs ever lower. Big investors will gain the most, of course. But retail investors, who often hold shares in mutual funds, will fare better too. High commissions and inefficient trading can double the typical fund's expense ratio--a cost that can run into billions of dollars a year for fund investors collectively. Mark Madoff, a veteran trader, believes that a flat electronic playing field is good for everybody. "There's very little evidence across the world that a competitive market is not superior to a monopolistic market," he says. "If we break down the barriers, the little guy will win."

For all the high-minded TALK about making the market more open and fair, the nitty-gritty of trading rules is being hashed out between dozens of constituencies, none eager to give up its interests. It's a messy tempest to get in the middle of. No wonder many Wall Street veterans were shocked to learn that John Thain would take the CEO job--leaving his comfortable, elevated perch as president of Goldman Sachs.

NYSE chairman Reed's first overture to Thain last fall was met with a polite no. Undaunted, Reed tried again three weeks later at a luncheon for German Chancellor Gerhard Schroder. "He started talking to me about the role of the exchange in American history and how important it was to the U.S. financial system," recalls Thain, who took a $16 million pay cut to leave Goldman. (Don't feel too bad--the NYSE job still pays $4 million a year.) Thain, a Goldman partner when the firm went public in 1999, also has shares worth about $300 million.

In the six months he's been on the job, Thain has been hustling, holding meetings with every conceivable faction. His proposed rule changes for the NYSE could come in just days or weeks. The SEC's reform plans for all the nation's stock exchanges will likely come, Donaldson says, by year-end. And then the NYSE will have to spend big on the technology to transfer trillions of dollars worth of share-trading from man to machine. The metamorphosis could take a while. But it will come.

Leaning back in his chair in his new office at the NYSE, Thain starts to talk about another special project he's been working on--the beehive he's put together with his son. The cold winter was not kind, killing off many of the bees, so the two recently ordered a new batch. "They arrive in the mail! Can you believe it?" he exclaims boyishly. "The mailman was carrying it waaay out in front of him."

When asked if he worries about getting stung, Thain's response is telling: "The trick is not to move too quickly. If you move too aggressively, you'll rile them up." Words to live by at the New York Stock Exchange.