Mortgage Rates 30 yr fixed 3.80% 15 yr fixed 3.20% 5/1 ARM 3.84% 30 yr refi 3.82% 15 yr refi 3.20% Find personalized rates:   Rates provided by Bankrate.com.

SALEM, Ore. (CNN/Money) - When Joe and Terie Wehage of Portland, Ore. bought a vacation house with another couple ten years ago, their goal was to find a retreat that could accommodate a growing family – they have five children between the ages of 8 and 18 – and bring in some rental income.

"It paid for itself pretty quickly," said Joe of the five-bedroom house in Sunriver, a golf and ski community about 200 miles southeast of Portland. So much so that a few years later the Wehages bought their partners' share of the property and started shopping for another vacation home, this time on the Oregon Coast.

"We realized that a large rental house was a great niche to have," said Joe, who is a mortgage broker and president of Foundation Mortgage. Terie is a stay-at-home mom, though managing the family's vacation houses is practically a part-time job. They now own five of them.

It isn't that the Wehages are fabulously wealthy. Rather, they're among a small but growing segment of the market that is buying multiple vacation properties for their own recreation, short-term rental income and the promise of long-term appreciation.

"They're typically at the end of their career, making a good living and have more control over their work schedule so they have the free time to spend at their houses," said Robert Moulton, president of Americana Mortgage Group in Manhasset, N.Y.

Moulton's clients generally work and live in the New York metropolitan area, already own a second house in the Hamptons or Vermont and are now looking to buy another house down in Florida.

"I'd say the third home is more of an investment play than anything else," said David Hehman, CEO of EscapeHomes.com. "People are buying in vacation destinations because they know the demand is there."

The second-home market is difficult to track because, on paper, second-home transactions are no different than those for primary residences. But it appears that areas where vacation properties represent a significant share of the housing stock have appreciated at a faster pace than other markets.

To get at just how strong vacation-home markets are, Fiserv CSW senior economist David Stiff paired 2000 Census Bureau statistics on seasonal housing units with price appreciation data for about 200 counties.

In the 14 counties where at least 10 percent of all single-family homes are seasonal, the median price appreciation was 59 percent between the end of 2001 and 2004. In counties where seasonal homes represented less than 10 percent of housing, meanwhile, the median appreciation was only 33 percent during that period.

In Monroe County, Florida – the Florida Keys – where about 24 percent of all houses are seasonal, property values increased 117 percent over that period. In Cape May, N.J., where 48 percent of all houses are seasonal, home prices increased 81 percent.

If you're shopping for vacation property, you'll want to make sure there is demand for short-term vacation rentals in that area, assuming you plan to rent it out.

"I would talk to people in the area and find out what the typical occupancy and rent are," said EscapeHome's Hehman.

The Wehages have been able to rent their newer properties about 100 days a year, said Joe, while their older property rents out 150 to 180 days a year thanks to repeat customers and word-of-mouth recommendations.

If you plan to hire a property manager to market and maintain the property, budget 20 to 30 percent of your expected rental income, said Hehman. The Wehages save on this expense because Terie manages the property herself, but they still have to pay for marketing, cleaning and maintenance.

Before you get too excited about a house, make sure that a neighborhood's homeowners association doesn't prohibit short-term rentals, as many do. The local government may even have an ordinance that prohibits such rentals.

In the past, second- and third-homes required larger down payments and were financed with higher interest rates. Today, according to LendingTree president Anthony Hsieh, vacation property is relatively easy to finance, assuming your credit is good and debt load within reason.

"Buyers have as much choice as they do buying their first home," he said.

If borrowers plan to rent out their property, banks will factor that rental income into the equation. "They will look at whether vacation rentals are common in the area and underwrite the loan as they would any other rental property," said Hsieh.