NEW YORK (CNN/Money) - A survey finds affluent Americans growing more concerned about the state of the economy.

There has been a sharp drop among those surveyed who expect further gains for the S&P 500, and a majority now believe there is a bubble in housing prices. A growing concern about the federal budget deficit now has a strong majority preferring deficit reduction to further tax cuts.

The quarterly survey also found the economy was cited as being on par with terrorism as a national concern for the first time in six quarters.

The survey was conducted by McDonald Financial Group of 400 affluent Americans with investable assets of $500,000 or more, and/or personal annual income of $150,000 or more. The late March/early April survey found that its overall consumer confidence index fell to 50, down 5 points, or 10 percent, from the 55 reading in early January.

The survey found growing concern about the federal budget deficit, with 85 percent of respondents viewing the deficit as a problem and 57 percent seeing it as a "very serious" problem. In fact, by better than a two to one margin the affluent Americans surveyed said they would prefer deficit reduction measures instead of tax cuts.

The survey found that 60 percent of those surveyed now believe a bubble exists in real estate prices, the highest reading since the survey started in January 2003. But while 26 percent believe the bubble will burst in six months to a year, 33 percent say it will burst in one to two years and 28 percent predict the bubble will burst in two years or more. And 27 percent still expect to make a major home improvement over the next three months.

Even with the growing concerns, 76 percent told the survey they plan to hold their luxury spending steady, and another 10 percent intend to increase that spending. And 48 percent still think the S&P 500 will rise over the next quarter, but that's down from the 60 percent who thought so in January.

"The affluent respondents that we surveyed believe that the Fed will increase interest rates, that real estate speculation will curb, and that stock market growth will slow," said David Legeay, senior vice president, McDonald Financial Group. "As a result, many affluent individuals are now paying down debt and focusing on long-term investments."

For more on the economy and what it means for you, click here.