NEW YORK (CNN/Money) - If you're being deafened by the sound of hammering and sawing in your neighborhood, as one or five or 20 new houses are built around you, you might think the U.S. housing boom is stronger than ever.

Well, it may be, but it's not going to last much longer.

Though the nation's housing market is unlikely to fall off a cliff, many economists believe it's probably at or near the peak that prices have been climbing for the past few years.

"We expect home sales will drop marginally -- about 3 percent -- in each of the next two years from this year's level, which will be an all-time record," said Douglas Duncan, chief economist at the Mortgage Bankers Association of America.

The reason for the MBAA's tempered outlook is the sudden, dramatic upswing in mortgage rates. After tumbling to record lows in June, rates accelerated in July and early August and are back near their highest levels in a year.

While rates still are low by historical standards, the turnaround in rates has pushed many potential buyers out of the market, heralding the end of this year's mad rush to the market.

Some economists think stronger economic growth -- which most forecasts say is right around the corner -- will help lift demand for housing, but Duncan said he doubted that would be enough to offset the impact of higher rates.

Related stories Housing starts jump Homebuilder optimism high Rates dip after 5-week rise Hot housing markets, 2Q: Everywhere Your Home: Worst-case scenario

"A fair number of first-time home purchases were brought forward because rates were so good, and a number of people also moved up in the market, when they were able to get a bigger mortgage at a lower rate," Duncan said. "When rates move up, you won't get those first-time sales or the move-up factor. Is that offset by the benefits of economic expansion? In our view, not quite."

In addition, some speculators who were hoping to make a killing in the home market might move on to greener pastures, including the stock market, when they see home-price gains slow.

What's more, though household income has been climbing slowly and steadily, despite the longest stretch of labor-market weakness since World War II, it can't compete with the abrupt jump in mortgage rates, driven by a 42 percent surge in the 10-year Treasury yield since mid-June.

"The indicator that economists use to predict the housing market is the affordability index -- the ratio of monthly income to monthly mortgage payments," said Ethan Harris, chief economist at Lehman Brothers. "As the economy picks up, the typical family's monthly income will slowly accelerate, but the monthly mortgage cost is quickly jumping, much faster than income could possibly move."

Housing has been so strong for so long that some economists have occasionally fretted that home prices were expanding into a dangerous bubble, similar to the bubble in technology-stock prices that popped in 2000, soaking investors.

But most economists dismiss that idea, and the coming drop in home sales and home prices likely won't come all at once.

Though the Commerce Department said homes were built at the fastest pace in 17 years in July, there's still a relative shortage of supply, and permit issuance -- a more forward-looking indicator -- did slack off in July, a hint of some sobriety on the part of homebuilders.

"The market has some adjustment to do, but it will be more of a slow leak than a pop," Harris said.

The lingering effect of this summer's earlier low rates -- including the rapid pace of homebuilding, as well as increased sales of furniture, appliances and other home and garden supplies -- likely will boost economic growth in the third quarter, many economists said.

After that, however, with the refinancing boom ancient history and home sales slowing, the housing market, which has basically carried the weak economy on its back in recent years, will have to be relieved of duty.

"The economy is picking up from a pathetic pace to something better, but the big question is what happens next year, when there's no more tax cuts and mortgage refinancing to boost growth," said Rory Robertson, interest-rate strategist at Macquarie Equities (USA).

"Will the jump in growth now kick-start a whole production-income-spending cycle, so that we get a self-sustaining expansion? That's an open question."