NEW YORK (CNNMoney.com) -- The housing markets in New Orleans and other Hurricane Katrina wracked areas are returning to normal, but very slowly.

The restoration effort in New Orleans is being slowed by a labor shortage, according to Chris Inman of TEC Realtors in New Orleans, speaking at a Coldwell Banker panel discussion this week.

It's a vicious cycle. Workers can't move there because there's no place to live; housing supply won't increase until workers put up more units. "It's going to take, probably several years to handle," said Inman.

Most of the brokers participating in the discussion reported that prices for undamaged houses shot up immediately after Katrina but have settled down again to about 15 percent to 35 percent above pre-storm levels. It would be even worse had some traumatized residents not chosen to move away.

Inventories are, of course, still tight. Areas that escaped the worst damage have absorbed many evacuees, supporting home prices.

Pearl River County, Mississippi has doubled in population, reports Bruce Kammer, of Coldwell Banker County Properties in Picayune, Mississippi. The area, which had been averaging about 425 properties in inventory, soon had fewer than half that. The count has rebounded to about 275 today. The average house price before the storm was $91,000 and today it is somewhere between $135,000 and $150,000.

Houston absorbed a huge influx of evacuees and handled it fairly well. More than 54,000 new home starts are expected in Houston in 2006, according to the Greater Houston Builders Association.

Steve Barnes, of Coldwell Banker United, said that Houston's single family house prices are up about 7 percent to a median of $156,000. The metro area still has an inventory of about 40,000 homes, down only about 45,000 from pre-Katrina levels.

Post-Katrina Baton Rouge went from 450,000 to more than 800,000 residents almost overnight, before draining down to 600,000, said Kim Ashford, an agent with Mackey and Company. Houses listed for sale there don't last long and usually draw multiple offers.

Carlene Alfonso, of Alfonso Realty in Biloxi, Mississippi also reported multiple offers for most non-flooded homes.

She added, "People are fed up and angry at insurance, at FEMA, at politicians, and at regulations. . . . On many occasions, we sold a property that had a FEMA trailer in the front yard, and it's taken us up to three months to get the empty FEMA trailer removed, and it's been a cause of some of our house sales falling through."

Four of the brokers in the discussion had lost their own homes. And Jon Ritton of Coast Delta Realty in Diamondhead, Mississippi, reports that cracks are starting to develop in the area's social fiber. "There's more drinking. There's drugs. There's mental health issues."

Meanwhile, Ritton says, home values in Diamondhead shot up immediately after the storm but have settled back down to 20 pecent to 35 percent above pre-Katrina levels as new homes have come on line. And many area residents decided they had had enough and had decided to relocate, further easing demand.

Many of the homes these people have vacated have been sold to people living in FEMA trailers for most of the past year.

"If you've ever been in a FEMA trailer, you know that's long enough," he says.

• Another storm is nearing the Gulf Coast and oil prices have jumped.

• Home Depot is finally back in New Orleans.
• Home insurance has become a huge problem for Gulf Coasters