BEND, Ore. (CNN/Money) - For now, mortgage rates are still near historically low levels. But more and more, economists are beginning to worry about a resurgence in inflation and thus rates. That, of course, would remove one of the key underpinnings of the recent boom in real estate.

To be sure, some economists have been incorrectly predicting higher rates for more than a year now. But now that Fed chairman Alan Greenspan has publicly declared higher rates inevitable -- without saying when -- it seems wise for homeowners to be thinking about the prospect.

Interest rates on a 30-year fixed-rate mortgage, the most popular U.S. home loan, averaged 5.89 percent for the week ending April 15.

But what happens if it starts ticking up more quickly?

All things being equal, home prices could suffer when rates go up. After all, buyers who could afford a $1,400 monthly payment on a $250,000 mortgage when rates are 5.5 percent may not be willing to pay as much for a house if rates go to 7 percent, in which case the monthly payment would be $1,660.

"Let's imagine rates go to 7 percent, I think that would take the air out of the bubble pretty quickly," said Dean Baker, co-director of the Center for Economic and Policy Research,

Other economists say the relationship between home prices and interest rates isn't quite as direct. For one, when rising rates go hand-in-hand with an economic recovery, as they often do, better job prospects partially offset the effects of higher rates.

"If rates rise it's because the economy is getting better," said JP Morgan Chase senior economists Jim Glassman during a recent National Association of Home Builders press conference.

Also, when rates go up, buyers often opt for adjustable rate mortgages (ARMs), which have lower rates than fixed loans. "When rates started picking up after the last refi boom in 1993, people didn't leave the market, they just shifted into ARMs," said Eric Belsky, executive director of Harvard's Joint Center for Housing Studies.

(For more on how ARMs work and whether they're right for you, click here.)

In March, the Mortgage Bankers Association said it expects the 30-year fixed rate mortgage to reach 6.2 percent by the end of 2005. But, favorable demographics and a growing economy will continue pushing up home prices, according to MBA chief economist Douglas Duncan, albeit at a slower rate than they have risen in recent years.

The median price of an existing single-family home is $172,200 as of the first quarter of this year, according to the MBA. At this time next year, the MBA is predicting, the median will be $177,400, a 3 percent increase.

As Baker sees it, however, even a slowing of price appreciation could have repercussions. "People buying in bubble areas are going to look at a house in a different way if they don't think its price is going to go up at 15 or 20 above the rate of inflation," he said. "Before you buy, ask yourself whether it's still worth it even if home prices come down."

Though there's never been a nationwide decline in real estate prices, individual markets have suffered plenty -- see "Real estate horror stories."

As such, some markets could surely feel more pain than others. "I'd be most concerned in places where housing affordability is an issue because the effects of rising interest rates are even more pronounced," said David Stiff, director of economic research for Fiserv Case Shiller Weiss.

According to the National Association of Realtors affordability index, Boston, San Diego, San Francisco and Washington D.C. were among the least affordable markets as of November 2003.

Among the most affordable places are Buffalo, N.Y., El Paso, Tex., Fort Wayne, Ind. and Peoria, Ill. In these cities, median-income families make more than enough to pay for median-priced homes.

"In cheaper markets interest rates probably won't matter as much as the local economy," Stiff added. "These are places where new supply matches new demand, and you just have steady appreciation."

Similarly, not all homeowners will suffer the same. If you're not planning to move for years, a decline in home values won't have as much of an impact.

But a decline could be a real problem for Americans who have taken advantage of the run up in prices to do cash-out refinancings. They could very well owe more than their house is worth -- bad news if they are forced to sell.