NEW YORK (CNN/Money) - Housing starts held steady in May, the government reported Thursday, as the reading on the strength of the real estate market came in below Wall Street expectations.

The Census Bureau reported that housing starts hit an annual pace of 2.01 million new homes in May, up 0.2 percent from the rate in April. Economists surveyed by Briefing.com had forecast starts at a 2.05 million pace in May.

The number of building permits, a sign of builder's confidence in the housing market, slowed in May. The government report showed an annual pace of 2.05 million permits in the period, compared with a revised 2.15 million rate in April. Economists had forecast a slower decline in the rate to 2.11 million in May.

While there was little change in overall housing starts, single family home starts posted a nearly 5 percent gain to an annual pace of 1.7 million. But the pace of starts for apartment buildings or condos with five or more units fell 19 percent in the period to an annual rate of 266,000.

Most of the slowing in building occurred in the South, the region responsible for roughly half the nation's home building. Single family home building fell nearly 8 percent, and multi-family housing starts fell by almost a third.

The Northeast and Midwest actually saw multi-family home starts stay roughly level, while the West saw only a modest decline in those starts, as single family home starts picked up. Overall single family home starts outside the South was up a solid 18 percent compared with April.

National Association of Home Builders Chief Economist David Seiders said that even though the numbers showing decline -- multi-family building and home building in the South are coming off very high numbers in April -- that the May numbers still look solid, especially as mortgage rates stay low.

"Worrying about any fundamental weakening is well down the track some where," said Seiders.

And while some of the comparisons to April suggest weakness, the year-to-date numbers for the first five months show solid growth compared the same period last year, except nationwide mulitfamily starts and starts in the Midwest, which were down only marginally.

Still a couple of independent economists said that the report suggested that the boon in home building could be topping out.

"Single family starts will likely remain elevated until we see a significant increase in long term interest rates," said Gina Martin, economist with Wachovia Securities.

The average interest rate for a 30-year fixed rate mortgage stood at 5.56 percent last week, according to mortgage lender Freddie Mac, the lowest rate of the year and down from the 5.72 percent average for May.

"Housing may get another mini boost from the recent drop in rates. These data do lag a bit," said Robert Brusca of FAO Economics. "Still, it is clear that, low rates or not, housing is not on fire the way it once was. The level of activity remains quite high for housing. But the prospects for further growth do not look that strong based on momentum."

For more news on the real estate markets and what it means to you and the economy, click here.