NEW YORK (CNN/Money) - She had to eat raw insects to do it, but Tina Wesson finally got to pay off her mortgage.

Fans of the television show, "Survivor," will remember Wesson as the Tennessee mom who out-smarted her competitors last year to win a $1 million jackpot, then announced she was using the cash to pay for her home once and for all. In fact, Wesson also was going to pay her best friend's mortgage, too.

What homeowner didn't envy Wesson?

Paying off your homestead is hardly the impossible dream -- even without jackpot winnings. By paying your mortgage in bi-weekly installments, for example, you can shave thousands of dollars -- and many years -- off your loan. For more, see "Save big on your mortgage." It also becomes possible for many to pre-pay their mortgage after coming into an inheritance or landing a fat bonus at work.

But tread with caution if you plan to follow in Wesson's footsteps. There are drawbacks you may not have considered.

Before you pay off that tax-friendly home, first determine what it is you'll lose. With interest rates on 30-year fixed mortgages hovering at around 7 percent, home loans are relatively inexpensive in the scheme of things. They also offer significant rewards in the form of tax deductions. Any interest you pay becomes a write off (assuming it's greater than the standard deduction), helping to lower the cost of that loan further still.

For example, a 7 percent mortgage for someone in the 27 percent tax bracket would really cost 5.11 percent after deductions. (You can calculate your rate by subtracting your tax rate from 100, then multiplying the answer by your mortgage interest rate. Multiply that answer by .01 percent.)

Once you've got a grip on costs, it's time to consider a slew of other factors -- from your age and work status to the kinds of financial goals you've yet to attain. Here's a rundown:

Strategies: Home Buying       Mortgages fall for 4th straight week How much home can you afford?

If you're youngish -- say, 40-years-old -- and you receive a substantial inheritance or land a fat promotion you'll no doubt entertain the idea of a mortgage-free existence. You'll lose the tax breaks, you reason. But you'll gain years of extra savings power.

Not so fast.

Before you pay off your house, make sure you've reached all your other financial goals. Your top priority should be stashing away enough for retirement. To figure out how much you'll need to while away your golden years playing tennis or cruising the world, use our "Retirement Planner".

If your employer offers a 401(k) plan, make sure you're contributing the maximum amount allowed. (This year, that's $11,000 for employees, and $12,000 for those over 50.) If you've been unable to deposit the max in the past, use your windfall to supplement your expenses so you can plow as many pre-tax dollars from your paycheck into the 401(k) as possible, said Dee Lee, a Certified Financial Planner and co-author of "The Complete Idiot's Guide to Retiring Early."

Once you've got your 401(k) fully stocked, look at other tax-sheltered savings tools, such as traditional or Roth IRAs. (This year, you can put in up to $3,000, or $3,500 if you're over 50.)

Next move on to second-tier financial goals.

If you have kids who want to go to college and you've fully funded your retirement accounts, you also might take your windfall and drop it into a state-sponsored 529 college savings plan. In fact, 529s let you put in up to $55,000 in a single year per child tax-free as long as you make no other contributions to that fund for another four years. Earnings in 529's can be withdrawn tax-free if they're used for qualified higher education costs. Some states even offer a tax break to residents who contribute to their plans. (You don't need to live in the state that sponsors a 529 to use one.) For a full rundown on various state plans, see the College Savings Plan Network.

If you're still working -- and well on the way to meeting your other financial goals -- pat yourself on the back. But hold off using any extra cash to pay off your mortgage until you explore other investments.

If you've got time on your side, then you'll likely do better in stocks or mutual funds. Over the long haul, the market has historically done better than the 5 or 6 percent you'll likely save by paying your mortgage early.

You'd be foolish to take money out of a 401k to pay the mortgage early. David S. Rhine Director, family wealth planning, Sagemark Consulting

The Vanguard 500 index fund, for example, has earned an average 13.36 percent a year since its inception in 1976.

Finally, even if you don't need to invest, have met your financial goals and appear to have all the money you'll ever need, there's yet another reason to resist paying off your house: economic uncertainty. You may be feeling flush this year, but who's to guarantee smooth sailing for the next 20?

"If you go ahead and invest the money rather than paying off the loan, you've left yourself some liquidity," said Don Weigandt, vice president for wealth advisory at J.P. Morgan Private Bank. "If you pay off the loan, and you need the liquidity in the future, you don't know what the borrowing situation will be like. That's a second factor you take into account before paying off the mortgage."

So when does it ever make sense? Only if having a mortgage over your head keeps you up at night or you're about to retire.

When it comes to mortgages and retirement, financial experts agree the two don't mix. After all, no one wants to be saddled with a hefty bill when they're no longer earning a paycheck.

That said, if you're entering retirement with substantial income -- from a pension fund, Social Security, investments and savings -- and you can use some of your riches to get rid of the bill, by all means pay it off, Lee suggested.

"There's something comforting about being in retirement and having that home paid off," said Lee.

But there are other reasons you should consider wiping out your mortgage before you settle in for life in the leisure lane, as well. Namely, that the biggest perks of a mortgage -- tax deductions and home-equity loans -- cease to be factors for retirees, added Lee.

Here's why: Mortgages are structured so you pay mostly interest fees at the beginning of your term, and mostly principal toward the end. It's only the interest that you're able to write off. As such, older homeowners who have just a few years left on their loan are mostly paying principal so their tax deductions are slim.

"The principal and interest portions have completely switched positions (toward the end of the loan term)," said Keith Gumbinger, vice president of HSH Assoc., which tracks mortgage data. "The fact is, the tax benefit of a mortgage to offset income becomes moot after a while."

To find out what year you start paying more principal than interest, use the Home Buyer's amortization calculator at the HSH Web site.

Meanwhile, any retiree who hangs onto a mortgage in order to tap their home-equity may be in for a rude surprise. Quite simply, if you're not earning a paycheck, you won't qualify for a home-equity loan, Lee said, adding that you need an income to support the payments.

If you're looking to have access to extra cash in the future, you might consider taking a home-equity line of credit, which lets you draw on cash if you need it. Many lines of credits are interest-only for the first 10 years. You don't have to use the loan unless you need it, but it may come in handy if you're hit with big bills during retirement. Once you tap the line of credit, you can deduct interest on the principal up to $100,000. But expect to pay a fee -- usually a couple of hundred dollars -- to have a home-equity line in place for future use.

One more note of caution. If you're nearing retirement, it's a good idea to pay off your debts. But if you're planning to tap your 401(k) to wipe out that mortgage, you're about to make a costly mistake.

"If the choice is using a 401(k) -- especially if the employer kicks in matching funds -- you'd be foolish to take the money out of the 401(k) to pay the mortgage early," said David S. Rhine, regional director of family wealth planning at Sagemark Consulting.

That pot of gold sitting in your 401(k) comes with all sorts of strings attached. Cash out early and you'll owe federal and possibly state income taxes on earnings -- in addition to a 10 percent early withdrawal penalty if you're younger than 59-½.

One way to avoid the penalty is to start taking equal and substantial payments -- which will be based on your age and life expectancy -- for five years, or until you reach 59-½. But you have to be at least age 55. But once you start payments, you can't stop them, so you'll end up depleting your retirement fund faster than you expected. For more information, see IRS Publication 590.

For more details on mistakes that can derail a retirement see "401(k): the fatal flaw" and "Top 10 retirement bloopers."