NEW YORK (CNNMoney.com) - Home builders are growing concerned about an increasing number of cancelled new home orders, which experts say could be a sign of an underlying weakness in the recent run in home prices.

Specifically, the cancelled orders could be the latest warning sign that buyers who were turning to real estate as an investment, rather than for their own housing needs, are shifting out of real estate. And that could mean that in many hot markets, the air is about to come out of over-inflated real home prices overall.

1. 2. 3. 4.

Mortgage Rates 30 yr fixed 3.80% 15 yr fixed 3.20% 5/1 ARM 3.84% 30 yr refi 3.82% 15 yr refi 3.20% Find personalized rates:   Rates provided by Bankrate.com.

A survey recently conducted by the National Association of Home Builders of its members found one in 5 reporting more cancellations than six months ago, with 4 percent of the overall group saying the increase in cancellations has been significant.

"When you start to see cancellations, you really get worried," said David Seiders, chief economist for the trade group.

Typically, a downturn in a local economy -- particularly its job market -- can cause a drop in real estate prices and an increase in home order cancellations.

But the trade group's survey found only 15 percent citing job losses by buyers as a cause for the cancellations. The survey, which allowed the builders to cite more than one cause for cancellations, found 45 percent saying it was due to a buyer's inability to sell their existing home and a third citing the buyers not being able to qualify for financing at a time of rising mortgage rates.

But Seiders and others say a big concern is a factor not cited on the survey, the fear that cancellations are being driven by real estate investors who were ordering new homes with the intention of selling them quickly in a hot real estate market. And Seiders said many of the 72 percent of those surveyed not yet reporting an increase in cancellations are already worried.

Experts believe that the home buyer intending to live in a home is reluctant to cancel an order, even if the market seems to have softened. But an investor-buyer who more closely follows the local real estate market is more likely to cancel an order, even if they lose some deposit money, if they believe that the local market prices have fallen enough that walking away is more cost effective than buying and selling the home.

The flight of investor-buyers from the housing market and the increased cancellations could therefore push real estate prices lower in different markets.

"If you've overbuilt the market and sales get cancelled, you have to do something with the homes," said Seiders. "The incentives we're seeing builders offering are clearly designed to support prices and stop cancellations."

Thursday, luxury home builder Toll Brothers (Research) warned that it is seeing investors bail on some markets, and supply now greatly exceeding demand in some cases. Besides the increased cancellations, new signed contracts fell 21 percent compared to a year earlier.

"Speculative demand has ceased and speculators are now putting their homes back on the market. The result has been more supply than demand in some regions," said the company's earnings statement. "Markets such as metro Washington, D.C., which are sound economically and showing healthy job growth, will need to work through their excess supply before the imbalance once again tips in our favor.

Toll Brothers Chairman and CEO Robert Toll said the company's cancellation rate came to 8.8 percent of orders in the most recent quarter, up from a historic level of only 5 to 6 percent.

"At 8.8 percent we hope it's plateaued now," he said in response to a question during the company's analyst call.

And home builders stock analyst Alex Barron of JMP Securities said that Toll Brothers actually has much lower cancellation rates than the industry as a whole. He said some other smaller, privately-held builders have more lax rules about the amount of money a buyer must put down, and how easy it is for them to get the money back if they do cancel.

"They don't have a standard practice; the less money they ask, the higher cancellation rates," said Barron. "I would say for most builders, it's at least 25 percent today."

Barron also is concerned about the impact that investor-buyers could have on the new home market, and the real estate market as a whole.

"I would say that probably for publicly-traded builders, at least 10 to 15 percent of orders were from investor-buyers," he said. "Those are the people driving up the cancellation rates. This (the increase in cancellations) is another warning sign. I think it's something we'll have to deal with for the next two to three quarters until the market stabilizes."

For a look a closer look at the real estate market and what it means for you and the economy, click here.