Even by the standards of a wild and woolly industry, the oil market has been amazingly chaotic. After hitting an an all-time high of $77 last summer, the price of crude has plunged to near $52 a barrel. (On Jan. 18, after this story went to press, crude oil dropped below $50 a barrel in trading at the New York Mercantile Exchange, before settling at $50.50.)

In the new landscape of volatility, the advantage goes to the companies that can best deal with risk--and that means Wall Street. Investment banks like Goldman, Sachs & Co. (GS ) are doing land-office business selling customized hedges to oil consumers that need protection from the ups and downs of the oil market. And with 530 energy hedge funds today, up from just 180 in October, 2004, the financial sector's involvement in oil trading--and its ability to profit from movements in either direction--has never been greater.

But is Wall Street responsible for causing the volatility that it so enjoys? Some say yes. Peter C. Fusaro, co-founder of the Energy Hedge Fund Center, which tracks hedge fund involvement in the energy sector, argues: "Now we're seeing bigger price moves, $2 a day. We never saw that before."

For the most part, though, financial investors aren't causing long-term price gyrations; they're simply exposing the volatility that exists in the real world. With their rapid-fire buying and selling, the new breed of oil-market investors is causing prices to respond more quickly to actual developments. That's a good thing, since producers and consumers of oil have timelier information for making decisions about, say, drilling a new well. "The financial markets are really the messengers. They've created more transparency around the volatility that was always there," says Peter Hancock, co-founder of Integrated Finance Ltd., a New York-based strategic advisory firm.

The inescapable fact is that there is little excess oil so any perturbation in supply or demand can send prices soaring or plunging. And there are perturbations aplenty. Driving prices up are political instability in the Mideast, strongly rising demand from China, stormier weather from global warming, and a lurking fear that oil production may have peaked. Driving them down are new fields brought into production to take advantage of high prices, as well a warmer winter and the possibility of a price war inside OPEC.

The futures market is betting on prices to rise about $7 a barrel over the next two years, to $59. But no one has much confidence in this forecast. David A. Wyss, chief economist of Standard & Poor's (MHP ), sees oil averaging in the low-$60s long term, but he likes to give his price prediction with a self-deprecating disclaimer: "plus or minus $30."

For the time being, cheaper oil is unquestionably a welcome development for the world economy. Merrill Lynch & Co.'s (MER ) North American economist, David A. Rosenberg, estimated in a research note on Jan. 16 that the oil price decline would add 0.75 percentage points to U.S. economic growth in the first quarter. He said cheaper oil was as good as three interest rate cuts by the Federal Reserve. But in a larger sense, the volatility of energy prices imposes a real cost by disrupting decision-making. Oil companies, automakers, airlines, and manufacturers all need to have some idea about the future price of crude to run their businesses efficiently. And investment in the alternative-energy sector is exquisitely sensitive to long-term forecasts of oil prices.

On the bright side, there are ways to live with volatility. The vastly increased volume of speculative trading in oil futures has increased market liquidity. The amount of unsettled oil contracts or "open interest" on the New York Mercantile Exchange--a measure of the volume of trading--hit a record 1.28 million barrels in early January, according to the Commodity Futures Trading Commission. In the past, "only those who could physically store large quantities of oil had the ability to trade," former Fed Chairman Alan Greenspan said in congressional testimony last year. But now, he said, "the new participants, investors and speculators," were "hastening" the market's adjustment to tighter supplies.

Firms like Goldman Sachs are expert at taking on risk from those who wish to shed it, carving it up, and selling the pieces. In the 1990s, Wall Street was encouraging energy companies like Enron, Mirant, and Dynegy to develop in-house trading expertise. But after Enron blew up in 2001, the investment banks "did a 180 on the industry and said, "Why are you guys taking these untoward risks?'" says Stephen Schork, editor of The Schork Report, an energy newsletter. Energy trading rapidly moved from Houston to the financial centers of Manhattan and Greenwich, Conn.

For nonfinancial firms, meanwhile, living with oil price volatility means either buying the protections that the financial sector is selling or finding their own way to defray risks. General Motors Corp. (GM ), for example, is hedging its bets without Wall Street's help by playing both sides of the street. It's churning out gas-guzzling sport-utility vehicles (in hopes of cheap oil) while simultaneously developing the fuel-sipping Chevrolet Volt, a hybrid car that you plug into the wall at night. Says gm chief market strategist Paul Ballew: "We have to be ready if it's selling for $20 a barrel or if it's $60 a barrel."

HEADACHE
Easy to say; tough to do. Repeated gyrations keep blindsiding oil producers and consumers. Pity, for example, the poor finance folks at Northwest Airlines Corp. (NWACQ ), who thought it would be prudent to lock in the cost of jet fuel in case prices went even higher. Northwest, which is under Chapter 11 protection from creditors, hedged fully half of its fourth-quarter fuel needs at $65 to $79 per barrel of crude oil. That was way above what it would have paid if it had bought all its jet fuel on the spot market. UAL (UAUA ), AMR (AMR ), and Delta (DALRQ ) also put on hedges that ended up costing them money. AMR CFO Thomas W. Horton says fuel hedging was about break even for 2006, producing gains in the first half and losses in the second half. "Hedging is one of those things you have to be careful with," Horton says.

Volatility is a huge headache for oil companies as well. John Felmy, chief economist of the American Petroleum Institute, says: "It really gives one the willies. You're making bets on investments that are going to run 30 or 40 years." Stefano Cao, general manager for exploration and production at the Italian oil giant ENI (ENI ), laughs that the company doesn't have an oil-forecast czar: "He would be fired every month. The rule of the game in our industry is whoever tries to gauge and decide on projects based on a specific oil price is going to be wrong by definition."

Fear of being caught by a sudden price drop makes oil producers highly risk-averse. In bidding for oil leases or buying smaller companies, most of the majors set a "hurdle rate" of $25 or $30 a barrel, meaning their investment has to pencil out as profitable even if oil prices get that low, says Nariman Behravesh, chief economist of Global Insight Inc. Companies like Exxon Mobil Corp. (XOM ) may not even entertain an acquisition when oil prices are above $40 because potential targets would be emboldened to ask too much.

But even for conservative oil giants, the existence of a deep, active financial market is a plus. The collective wisdom of the world market produces a view of the future that their own internal planning staffs can't match.

It's not just PhD quants or highly paid traders who are affecting world oil markets. It's the little guys who have started to put, say, 5% of their retirement money in a commodity fund. More than $100 billion is now invested in commodity funds that track the Goldman Sachs Commodity Index or the Dow Jones-AIG Commodity Index.

Not satisfied with simply trading in the futures market, some financial firms are getting their hands dirty by buying physical assets such as pipelines and terminals, as Enron once did. This gives them a perfectly legal inside look at the workings of the energy market. The Fed fueled the trend in 2003 by allowing commercial banks to take possession of physical assets. Goldman Sachs has a stake in a Kansas oil refinery and a natural gas pipeline running through New York and Connecticut, among other holdings. In August, Kinder Morgan Inc., (KMI ) a Houston-based energy transportation and distribution company, said it was being acquired by a consortium led by Goldman Sachs, AIG (AIG ), the Carlyle Group, and Riverstone Holdings in a deal valued at $22 billion. "Having a direct and intimate involvement in the asset enhances our access to information" such as where the bottlenecks are, says Paul Addis, co-CEO of a new venture between Highbridge Capital Management, a New York-based hedge fund, and commodities house Louis Dreyfus Group.

But this kind of special access is coming in for criticism. In a June, 2006, report, the Senate permanent subcommittee on investigations concluded that such "excessive speculation" in oil by hedge funds and private equity groups has distorted market signals. "How much of the volatility is because of supply and demand, and how much from uncompetitive trading practices?" asks Tyson Slocum, director of the energy program at Public Citizen, a Washington-based consumer rights group. "We're glad to see prices go down. The problem is that the volatility isn't good for anyone."

Actually, it is good for traders. But don't blame the Street. The gyrations are real, and they're here to stay. Jan. 9.