(MONEY Magazine) – SUMMMER'S WINDING DOWN, AND YOU'VE STILL got vacation on the brain: the quiet of the country, the crisp mountain air, the lull of the ocean. In the hope of holding on to those images a bit longer, you may have begun toying with the idea of a second home. You're not the only one. "A lot of people go on vacation, look at real estate fliers and suddenly decide they want to buy a house," says David Hehman, CEO of Escapehomes.com.

On the other hand, you know from your primary home's market that real estate is not the no-brainer investment it once was. And so you wonder: Are you letting your heart run away from your head? Should you even be considering a vacation home?

Probably not, unless you can accept that you're no longer going to get rich quick by buying a second home. Between 2001 and 2006, the median home price in the 30 top vacation markets rose 120%, according to David Stiff, senior economist for Fiserv Lending Solutions--nearly twice as much as the appreciation in primary markets. "Prices shot up because supply couldn't respond," says Stiff. In those glory days, it was possible to flip a property for a six-figure profit in a matter of mere months, and the feeding frenzy of investors pushed the median ever higher.

That has left second-home prices even more vulnerable to a downturn, says Stiff, because demand doesn't hold steady; nobody needs a second home. Most house-sale numbers for this spring aren't in yet. But in the first quarter, "sales volume in vacation areas dropped a lot, " Stiff reports. For a prime example of how quickly markets can turn, consider Naples, Fla., which showed a nearly 40% appreciation between the fourth quarters of 2004 and 2005. Yet between June 2005 and June 2006, sales volume dropped 48%, according to the Florida Association of Realtors. And median prices, meanwhile, fell 8%.

Don't Buy to Invest

Michael Costa, a 39-year-old equity trader in New York City, can tell you all about it. In July 2004 he bought a house in Naples, largely because it was one of the hottest markets he could find. By the time he listed the house in fall 2005, the market was already deflating. The property took 10 months to sell, and he got 19% less than his original asking price. Costa made money, but not nearly as much as he would have months earlier. "I feel lucky because I think it's going to get worse," he says.

So far, however, Naples' rapid decline seems to be an anomaly. Like primary-home sellers, second-home sellers have been loath to drop prices, Stiff says, preferring instead to keep their houses on the market longer. As a result, prices in many vacation areas have continued to rise, though at a slower pace. In Cape Cod, prices were up only 3% between the first quarters of 2005 and 2006 compared with 5% the prior year. Lake Tahoe's El Dorado County was up only 7% vs. 26% the year before.

That's a sobering trend if you were counting on rapid appreciation to justify investing in your favorite garden spot. In that case, "you're probably going to be disappointed," says Nicolas Retsinas, director of Harvard University's Joint Center for Housing Studies. "The past several years of appreciation were unprecedented but clearly weren't sustainable."

Do Buy to Own...

It's not all grim news. The moral of this market: If you are going to buy, do it for love, not money. Do it because it's a place you adore and will want to use for years to come. That way, even if prices dip--or plummet--you will be able to enjoy it while you wait out the fluctuations. "Right now you should be thinking consumption rather than investment," affirms Harvard's Retsinas.

By that token, if you've always dreamed of owning a piece of oceanfront or having a slope-side condo for ski season, you may actually find today an appealing time to buy. For one thing, the deflating bubble has shifted the power in the market from seller to buyer. Gone are the days of bidding wars and hasty offers. In most markets, houses are taking longer to sell, and consequently owners are offering incentives and showing a willingness to negotiate. "If you find a house you like, you can almost watch for it to go down in price," says Hehman. The glut of properties owned by investors also bodes well for bargain shoppers, he says. "As time goes on, their tolerance for holding things is going to change."

Meanwhile, the rental market is recovering from a prolonged slump brought on by the ownership craze. Rising interest rates have made renting more feasible than buying for many. Rents are rising in a number of markets, and occupancy rates are at five-year highs. So while you'll pay more to finance a home, you may have better luck renting it than you would have had a year ago.

...But Buy Smart

Still, don't let the fresh air and margaritas kidnap your common sense. It's important to be realistic, especially in an uncertain market. First off, make sure you truly like the community. For that, there's no substitute for spending time in a prospective town. "The more you know an area, the better you can eliminate some of the risk of buying," says Hehman. Rent for one season, and you'll also find out about up-and-coming areas and projects that could affect your investment.

Second reality check: Can you afford this? "The basic rule of thumb is that your housing costs--including those for your primary home--should be a third of your overall income," says Jay Mastilak, a certified financial planner and senior vice president for PNC Investments. Include mortgage, maintenance, property taxes, association fees, insurance and travel expenses. It's also worth noting that higher rates on mortgages and home-equity loans may cancel out the advantages of buying at a narrow price cut. So do the math.

The next thing to figure out is whether you'll spend enough time at the house to justify the expense. Come up with a back-of-the-envelope estimate by adding up annual expenses and dividing that sum by the total nights you'll spend there. Don't be surprised if you find it's cheaper to stay at a five-star resort.

But maybe you're thinking of offsetting the cost by renting? Make sure your work will be as easy as you're hoping: Consult property management firms about when, how long and for how much you could expect to rent the property.

You should also get a sense of the long-term sale potential in the community. While you may accept that your home won't sell for twice what you paid anytime soon, you don't want to lose money either. Ask local agents for appreciation averages for the past 10, 20 and 30 years; ideally, you'll see a positive number.

Once you've done all this homework, the fun part (read: open houses) begins. The bright side of this market is that you can--finally--take your time shopping for the right place. This year Birmingham residents Bill and Walker Jones, 55 and 53, began building a house near Big Sky ski resort in Montana after 10 years of vacationing nearby. Bill is sure they're making the right investment, and at the right time: "I think we'll look back and see this as one of the best decisions we've ever made."

SEASONAL HOT SPOTS

Vacation areas saw some of the biggest gains between early 2001 and 2005. Now many of these markets are beginning to cool.

NOTE: Based on county. SOURCES: U.S. Census for housing units; Fiserv Lending Solutions for appreciation.

More Value for Your Vacation Dollars

If you still can't afford the top markets, check out these five underpriced areas, identified by EscapeHomes.com

1 Central Sierra Mountains of California

Angels Camp, Arnold, Murphys, Sonora

Known as the gateway to the Sierras, this area offers skiing in the winter, plus golfing, mountain biking and hiking in summer. Homes start at $300,000. A 1,862-square-foot, three-bedroom house with an all-season spa and wooded views outside Arnold was recently listed at $388,900.

2 South Texas Coast

Laguna Vista, Port Isabel, South Padre Island

Here you'll find some of the most unspoiled beaches along the Gulf of Mexico. (The only spoiler is that this is also a prime spring break destination.) A house on the waterfront can go for up to $2 million; but you'll find better deals on inland condos, which go for $200,000 and up.

3 White Mountains in Eastern Arizona

Pinetop-Lakeside, Show Low, Snowflake

All the advantages of Aspen--hiking trails, fishing and, yes, skiing--without the clubby resort culture. You can get a two-bedroom, two-bath rustic cottage tucked into the woods of Pinetop for $259,000.

4 The North Fork Valley on Colorado's Western Slope

Hotchkiss, Paonia, Crawford

Artists, urban refugees and outdoorsy types have convened on this area, where desert meets mountain and New Age meets New West. Get a 2,504-square-foot, three-bedroom log house with mountain views for $325,000.

5 Northern Coast of South Carolina

Myrtle Beach, North Myrtle Beach, Murrells Inlet, Surfside Beach

This is one of the most popular vacation spots on the eastern seaboard, thanks to its pretty beaches, mild winters (average January temperature: 57°F) and southern hospitality. Consider settling along the 60-mile oceanfront area known as the Grand Strand; condos there start at $200,000.

SOURCES: EscapeHomes.com, Tenney and Associates GMAC Real Estate in Show Low, Ariz.; Century 21 Sierra Properties in Murphys and Arnold, Calif.; Lynne Tate Real Estate in South Padre Island, Texas; Bluewater Properties in Myrtle Beach, S.C.; Coldwell Banker, Colorado Realty in Paonia and Hotchkiss, Colo.

Can You Get a Tax Break?

A little help from the IRS can ease the expense of a second home

If you use the house for at least 14 days a year, you may be able to deduct the mortgage interest just as you would for your primary residence (assuming your total mortgage debt doesn't top $1.1 million). Visit less and you get no interest deduction.

If you rent it out for no more than 14 days a year, you won't owe taxes on your rental income. Rent for longer--or, if you stay there less than 14 days a year, rent it for any amount of time--and you will. But you can deduct expenses from rental income, and you may be able to write off losses against regular income.

home

Testing appraisal websites 61

Latest loan rates 61

Four "upgrades" that can scare buyers away 62

Additional Reporting By Asa Fitch contributed to this article.