(FORTUNE Magazine) - On a sunny May day in an office park in the Surrey countryside outside London, a Ferrari-driving hedge fund manager named Aref Karim is scooping up contracts to buy oil in July for $75 a barrel, $3 more than it's currently selling for. His company, QCM, has garnered more than $125 million in fresh cash since the beginning of the year, and he's itching to keep it at work.

A few hundred miles to the south in Paris, on the trading floor of Societe Generale Asset Management, Arnaud Chretien's team of Ph.D.s and engineers operates in near silence, with powerful computers buying and selling commodities according to preset algorithms, taking advantage of swift movements in everything from heating oil and gasoline to zinc and copper.

More from Fortune Will Mmmhops be a hit? NBA confirms L.A. Clippers sale to ex-Microsoft CEO Steve Ballmer FBI and SEC probe into Carl Icahn and golfer Phil Mickelson FORTUNE 500 Current Issue Subscribe to Fortune

What could turn the tide Plenty of things might change the direction of oil prices. The odds... Peaceful resolution of standoff with Iran: We can only hope. Cooling Chinese economy: Inevitable, but when? Increased production from Iraq, Nigeria: Don't count on it. Drop in U.S. gasoline consumption: If gas stays at $3, a definite possibility. New energy sources come online, from ethanol to new oilfields: It will take years. A drop in oil prices prompts traders and speculators to move on to the next hot area: That's the way it works.

How a market heats up Oil's price climb started well before the hot money moved in. Here are six crucial catalysts. Tank filling China's economy, growing at 10%, recently passed Japan as the world's second-largest oil consumer, while India and other developing countries are surging. In the U.S., by far the globe's No. 1 energy user, gas demand is up 1.7% annually over the past decade. Battle wounds Iraq's oil production remains below prewar levels, and saber rattling in Iran fuels uncertainty about supplies traversing the Strait of Hormuz. Profit focus Big Oil has been wary about over-investing in new production for more than a decade, fearful of hurting their margins if oil prices retreat. So cash that might otherwise have gone toward enlarging supply remains on the sidelines. Turf wars Unrest in Nigeria - including kidnapping of oil workers and pipeline sabotage - has cost roughly 500,000 barrels a day. Venezuela, where production is still feeling the effects of a 2003 strike, has begun raising taxes on foreign drillers. Saudi rumors The so-called Peak Oil theory, that the earth's supply of crude is dwindling, has been fed by worries that the productivity of Saudi Arabia's oilfields is slowing. Meanwhile, some major oil companies struggle to replace what they pump - Shell hasn't fully replenished its reserves for several years. Hurricane chaos Katrina & Co. swamped refineries and waylaid deep-sea rigs, disrupting supplies and driving up gas prices. Another potentially harsh hurricane season looms.

"It's quiet because we try to think," says Chretien, a dapper 41-year-old who sports an Hermes watch and resembles a younger, Gallic Kurt Russell.

Half a world away, near Los Angeles, the managers of the Pimco Commodity Real Return fund are deploying $12 billion into an energy-heavy index of commodities - holding millions of barrels of crude for retirement nest eggs the way an S&P 500 index fund owns GE and Microsoft.

Once upon a time, in the late 1990s, these billions might have been chasing Internet stocks. Now the hot money wants to be in energy. But this isn't just another investment fad. Because while no one was obligated to own Amazon or Pets.com during the tech bubble, all of us buy gasoline to drive our cars or heating oil to stay warm in the winter. Even if we were greener than Al Gore and heated our homes with peat and solar panels, we'd still have to pay higher energy costs being passed along by airlines and hotels and supermarkets.

It's no surprise, then, that the parallel rise of commodities trading and energy prices has sparked a fierce debate over whether hedge fund managers like Karim and other financial players are responsible for $3-a-gallon gas, with the blame game stretching from the boardrooms of Big Oil to Capitol Hill and Bill O'Reilly's Fox News studio in New York.

Indeed, just about everyone is ducking for cover. Big Oil is pointing fingers at hedge fund managers, who blame commodity index funds, who in turn cite surging demand in China, production losses in Nigeria and Iraq, and hostile regimes in Iran and Venezuela.

Fox's O'Reilly, at least, is clear: He blames all "these Vegas-type people [who] sit in front of their computers and bid on 'futures' contracts." As he puts it: "Supply and demand? - my carburetor, this has nothing to do with the free market."

If only it were that simple. The reality is that while many hedge fund managers and traders are making reams of money off of rising prices - Karim and his six-member team will share a multimillion-dollar pot this year if their stellar returns hold up - they don't have the heft to control the huge $2 trillion worldwide oil market. (Indeed, if all the hot money were in cahoots, then how is it that the average commodities hedge fund is, according to Merrill Lynch (Research) trading strategist Mary Ann Bartels, up just 5 percent year to date?)

That's not to say investment flows have zero impact on prices. As pension funds and investors reallocate more of their portfolios to commodities, the flood of assets has spawned strong momentum in the futures market.

But figuring out precisely who is responsible for the oil surge is complicated by long-term forces, such as years of underinvestment in exploration by Big Oil when prices bottomed out near $10 in the late '90s. Then there's the intense demand spurred over the past decade by that cheap energy in both developing economies and America's suburbs, where SUVs now run amok.

What the public hysteria over gasoline prices actually demonstrates is that the greatest capitalist country in history needs a refresher course in how free markets operate. That's especially true because market forces are amplifying price swings in ways we haven't seen before - and political leaders are weighing in with rhetoric that could only make matters worse.

Sure, commodity booms have come and gone (the Gold Rush of 1849, anyone?). But this is the first time that what's happening on trading floors in New York, London, and Paris is so closely intertwined with the lives of ordinary people.

"We've seen this with gold lately," says John Browne, CEO of BP (Research), referring to the precious metal's rise to a 26-year high of more than $700 an ounce. "Well, so gold jewelry becomes more expensive. When oil goes up, though, that is a big deal - a really big deal. It affects everybody."

The blame game starts with the big oil companies. Whenever he commutes between Royal Dutch Shell's (Research) London office and its headquarters in The Hague, CEO Jeroen van der Veer flies over the port of Rotterdam, Europe's energy hub, and notices the same thing.

"I see tanks full of product," he says. "The crude carriers aren't waiting to have to unload in the port. There are no lines at Dutch gas stations. Airplanes aren't waiting for fuel."

In other words, the current price run-up isn't like the shocks of the 1970s, when there were real shortages and drivers had to wait for hours to fill up.

"So if inventories are normal, why should the price be so high?" asks van der Veer. At least part of the answer, he concludes, must lie in the financial markets. "I know various pension funds that had money in bonds, in shares. Now they went into commodities," he says. "That's new. And that has to have an impact."

Joe Stanislaw, an independent energy advisor to Deloitte & Touche, calculates that the physical realities of supply and demand point to a price of $50 a barrel for oil, while geopolitical uncertainties in Iran, Venezuela, and Nigeria add another $10. Anything above that, he argues, is due to oil's popularity as an investment.

"Certainly the fundamentals are strong," says Stanislaw. "But the financial players exaggerate the upward movement by driving the trend."

It's an argument that neatly shifts attention from the enormous profits oil companies are making. Last year Exxon Mobil (Research) earned $36 billion - more than any company in U.S. history - and it added another $8.4 billion in the first quarter of 2006. (BP and Shell, for their part, earned a combined $12.5 billion last quarter.)

With even the oilman in the White House cautioning about "price gouging," the industry suffered a PR blunder when retiring Exxon CEO Lee Raymond walked away with a $100 million lump sum, on top of $250 million in restricted stock and options.

"It was like a red flag in front of a bull, and it came at the worst possible moment," says one industry insider.

Notes van der Veer, who earned a comparatively modest $4.3 million last year, "U.S. pay packages have to be judged in the U.S. market. European pay packages are more conservative."

Of course, just because oil companies are benefiting from the price boom, that doesn't mean they're causing it. The truth is, there is little they can do to immediately lower worldwide prices. They can hardly be expected to sell their product for less than buyers (like China) are willing to pay. And even if they did, the impact would likely be fleeting.

As van der Veer points out, his company accounts for just 3 percent of global oil production. The five supermajors - Exxon, BP, Shell, Chevron (Research), and Total (Research) - together account for only 13 percent of the 84 million barrels of oil the world consumes each day. By contrast, the state-dominated oil industries of Russia, Kuwait, Saudi Arabia, and Venezuela account for 30 percent.

"This isn't passing the buck," says van der Veer. "We're a small player in the global pond."

But it is also true that decisions Big Oil made years ago haunt the global market. Many of today's supply problems actually stem from the low prices we remember so fondly from the 1990s. Between the end of the first Gulf War and Sept. 11, 2001, oil prices averaged $20.75 a barrel.

Not only did that encourage demand in China and other emerging markets, says consultant Peter Schwartz of the Monitor Group, it also caused the supermajors to cut back on spending to develop new fields. "We're paying the price now for cheap gasoline a decade ago," says Schwartz.

Even today oil executives are reluctant to empty their coffers for new development, fearing that prices will trend lower in the future and hammer their margins. Yet that thinking serves only to increase the likelihood that supply will stay tight. It is also a train of thought financial players are keenly aware of - and more than willing to exploit.

Inside the fraternity of hedge fund managers and traders (women are still a rarity on the desks), the question of just how much impact they have on the oil market is something of a sore point. Unlike the floor traders shouting in the pits of the NYMEX, these are usually cerebral types - many have Ph.D.s in math and physics - who know the numbers cold.

"We're not directional gamblers," insists Julian Barrowcliffe, who runs the Anglian Commodities hedge fund. "We're not buying zillions of barrels of oil and sitting there hoping it goes up."

The British-born, New York-based Barrowcliffe started at Shell in London in the 1980s, moved over to Bankers Trust and then Merrill Lynch, and by the late 1990s was running Bank of America (Research)'s oil-trading desk. He struck out on his own in 2003 and now has more than half-a-billion dollars under management. A keen sailboat racer, at age 43 he's typical of the top commodity players.

"You've got to be young enough to handle it, old enough to have some experience," Barrowcliffe says.

His fund doesn't make huge wagers on the direction of oil - instead they make smaller bets on the spreads between different products in what traders call the energy complex. That might mean buying light, sweet crude while selling the heavy, sour stuff, or selling heating oil and loading up on gasoline ahead of the summer driving season. The key isn't how much one of these goes up or down - it's correctly anticipating the difference in price between the two.

The bets themselves may be small, but the rewards for successful traders are large. BP's top trader last year earned $16 million, $5 million more than CEO Browne. Barrowcliffe lives in a 2,700-square-foot loft in Manhattan's SoHo. Even younger traders are enjoying the fruits of this bull market, as banks like Barclays (Research) and UBS (Research) build their trading desks and raid talent from more established players like Morgan Stanley (Research) and Goldman Sachs (Research).

"We're doing five times as many hires per month as we did two years ago," says Justin Pearson, managing director of Human Capital, a London headhunter that specializes in the energy commodities field. "A good, midlevel trader can earn $1 million to $3 million today. Before, you had to be a top trader to do that."

It's easy to see why oil speculators are targets. "This is age-old stuff," says Craig Pirrong, an energy expert and professor of finance at the University of Houston.

When a currency crisis swept through Malaysia, Thailand, and South Korea in 1997, government officials blamed currency speculators, not their own misguided monetary policies. Back in 1958, U.S. onion growers were so convinced speculators were hurting prices, they actually convinced Congress to ban futures trading in onions - a ban still in effect.

Statistically, there is some correlation between rising futures volumes and rising oil prices (though it's hard to prove which is cause and which is effect). And there is at least one area where momentum buying by futures traders almost certainly has an impact. In the oil market, if futures prices are higher than the current spot ones - a condition commodity gurus call "contango" - that indicates a belief that spot prices will move higher.

That's where the market is today, with the price of July contracts for oil several dollars above the current price. Apparently, some traders are following the roughly $85 billion tracking benchmarks like the Goldman Sachs Commodity Index and the Dow Jones-AIG Commodity Index.

Like an 800-pound gorilla, when this index money moves, the whole jungle shakes. In early March, for instance, when the managers of the Dow Jones-AIG index switched from one type of NYMEX gasoline contract to another, wholesale gas prices dropped by nearly 9 cents a gallon.

"The index money dwarfs whatever the hedge funds do," asserts Barrowcliffe. "The notion that there are big bad armies of hedge funds creating this situation is very simplistic."

John Brynjolfsson is an unlikely scapegoat. A 42-year-old with a physics degree from Columbia and a master's in economics from MIT, Brynjolfsson runs Pimco Commodity Real Return. It is one of the biggest beneficiaries of the commodity craze, growing from zero dollars at its 2002 launch to $12 billion today. Rather than be defensive about the idea that booming funds like his are responsible for high oil prices, Brynjolfsson argues that energy investors deserve a pat on the back, not a kick the groin.

"There's no doubt in my mind that bringing capital into the commodities markets helps those markets work better," he says.

Brynjolfsson contends that what's needed to trim oil prices are economic incentives to create new supplies. He notes that a contango market allows oil companies to lock in high prices, mitigating the uncertainty of costly exploration projects.

"Industry insiders can invest capital in new production and then sell that production forward at higher prices," he says.

Contango markets also encourage the building of bigger oil inventories, given that oil stored today can be sold later at a higher price. While redirecting some oil into storage probably exerts a modest upward pull on prices, Brynjolfsson maintains that it's a worthwhile tradeoff for the economy.

"One reason gasoline prices reacted so violently to Hurricane Katrina is that inventories were low," he says. With bigger inventories as a cushion, the market is better prepared to cope with disruptions.

Most economists agree that robust trading dampens price volatility. Philip Verleger, a Colorado-based energy consultant, contends that if there were no futures market in oil, prices could be peaking at $90 a barrel rather than $75.

"You'd have a much higher probability of a spike," says Verleger.

That's also the opinion of the futures market's government regulator, the Commodity Futures Trading Commission. Michael Haigh, a CFTC senior economist, concluded in a report last year that price movement in oil is being initiated by commercial traders - i.e., oil companies, utilities, airlines, etc. - not money managers.

"Speculators," says Haigh, "have not caused the price increase."

Much of the backlash seems to stem from a basic misunderstanding of how futures markets operate - and how they differ from stocks. Commodity futures allow producers, consumers, and investors to hedge price risk by contracting to buy or sell a commodity at an agreed-upon price at a future date.

If prices were to move away from the consensus view - if, say, a majority of market participants believed that the July price of $75-a-barrel oil is unjustified given the price of gasoline futures (which is exactly the kind of bet favored by hedgies like Barrowcliffe) - there would be a rush to sell July oil, which in turn would push down the price.

Also, unlike stocks, which are open-ended financial instruments - meaning investors never have to take home a piece of the company they've bought shares in - in the futures market, oil traders must either deliver or take delivery of an actual product when the contract expires. Of course, the typical hedge fund or index fund sells the future before delivery has to be made.

"But," says Kevin Norrish, director of commodities research at Barclays Capital in London, "the price of futures is still inextricably linked to supply and demand in the physical market."

The other big difference between oil futures and equities - and another reason futures are less susceptible to speculative bubbles - is every futures contract requires both a seller who's betting prices will fall and a buyer who thinks they'll rise.

So for all the hotshot hedge fund managers making down payments on a Ferrari with their oil-trading profits, there are others who are in a world of hurt. (In the immediate aftermath of Hurricane Katrina, energy traders at Citadel Investment Group and Ritchie Capital reportedly incurred hundreds of millions of dollars in losses on account of their short positions in natural gas.)

One other point: The big index funds tend to buy futures that come due a month or two out. If they alone were driving prices, you would expect contracts further out in time to be priced lower. Yet even those contracts - dominated by commercial traders like oil companies and utilities - are in similar territory. (At $71 a barrel, the futures price of December 2010 oil is only $4 less than July 2006 oil.)

"To me, that's one of the strongest arguments that there's not a bubble," says Norrish.

So why then is oil so expensive? While financial activity plays some role, there needn't be anything nefarious about the motivations of these market participants, each of which is operating perfectly rationally.

The bigger answer to the oil price run-up seems to be the one nobody wants to accept: supply and demand. Because energy demand is so inelastic - consumers don't quickly change their consumption habits when prices go up - and new supply takes so long to come online (Exxon's multibillion-dollar Sakhalin-1 project took a decade to reach production), small changes can have an outsized impact on price, especially in a tight market. And that's exactly what's happening now with oil.

Barclays projects worldwide oil demand will rise 1.4 percent in 2006 with production up only 1.0 percent. The gap will have to be made up by drawing down inventories, which in turn will make oil markets more volatile. (Of course predicting future energy demand is no easy task. Just ask all the pundits - FORTUNE among them - who sounded the alarm over rising natural-gas prices last fall. Since December the price of natural gas has cratered 60 percent.)

The good news is that the solution to high oil prices is built right into the market itself. While near-term demand may be inelastic, the longer gas stays at $3 a gallon, the closer we get to a tipping point at which consumer behavior changes - carpools, anyone? - and the political resistance to coal and nuclear power gets whittled away. And the longer oil stays high, the more investment capital will pour into everything from ethanol to solar energy to new oil development.

That's not to say there won't be some pain along the way. But if you believe in capitalism, then you have to believe the market will push us in the right direction. So, if you want, go ahead and blame investors for what you're paying at the pump. Just remember to send them a thank-you note ten years from now when you plug in your first electric car.

Reporter Associate Doris Burke contributed to this article.

FEEDBACK jbirger@fortunemail.com, nschwartz@fortunemail.com