NEW YORK (CNN/Money) - Buying a home brings all kinds of joys -- the satisfaction of not paying a landlord, the pleasure of having your own space -- and best of all, the tax deduction.

Unfortunately, not all homeowners are taking advantage of this perk. Roughly one million taxpayers fail to take the deduction, losing $493 million annually, according to a recent General Accounting Office report.

Don't expect Uncle Sam to remind you about deductions when it comes to your abode. But we'll sort it out for you. Here are today's five tips.

Mortgage interest on your primary home is deductible unless your mortgage balance is more than $1 million or you used the proceeds of the loan for something other than buying building or improving your home.

CNNfn's Gerri Willis shares five tips on how to take advantage of homeowner tax deductions. Play video (Real or Windows Media)

To claim the deduction, fill out Schedule A, which is labeled "itemized deductions." Your lender will mail you a "Form 1098" that details your mortgage interest paid for the year. Late payments are bad for your credit but you can deduct those charges, as well as mortgage prepayment penalties.

Interest you pay on home equity loans or lines of credit secured by your primary or secondary home are also deductible up to loan amounts of $100,000.

Points are upfront charges a borrower pays to get a mortgage. One point equals one percent of the loan amount. You probably already know that the points you pay on your first mortgage to buy down the interest rate are deductible in the year you buy, but what if you refinance that home?

In that case, you'll be able to deduct only 1/30th of the points in each year of a 30-year loan. (And, for a 15-year loan, you'd deduct 1/15th of the points in each year of the loan.)

However, if you refinance two years back to back, you can take those write-offs from last year's refi all at one go since the loan is paid off.

An important tip comes from Bob Walters of Quicken Loans. He says that even if the seller pays the points for you, you can still reap the tax deduction. Sellers sometimes do this in order to sell their home without reducing the price.

This example comes from Walters: Let's say you're selling a home for $200,000. It hasn't sold yet, so you have two choices. You can lower the price by $10,000. Or you can offer to pay 3 points toward the buyer's mortgage, effectively lowering the rate on a 5-year ARM from 4.5 percent to 3 percent, or a 30-year fixed from 5.5 to 4.5 percent. Three points on a $200,000 loan is only $6,000, rather than the $10,000 you might have had to cut your price by.

Home sellers may avoid capital gains taxes altogether by paying attention to the exclusion available to them.

For owners that have lived in and owned a home for two of the last five years, single taxpayers can exclude gains of $250,000, while married couples filing jointly can exclude $500,000 of gains.

Mark Luscomb, a tax expert at publisher CCH, says that the IRS has been making it even easier to qualify for the exclusions. For example, a medical necessity (including giving birth to twins) or job change can be considered hardships that allow you to qualify for the exclusion.

If you're buying a home this year, you'll want to keep track of costs associated with buying a home. The reason? Someday you'll sell your castle and you'll need to know the basis, that is, the amount of your investment in property for tax purposes.

Uncle Sam wants you to lump in attorney fees, broker fees, title fees and survey costs into your purchase price for a total, or basis. Over time, you'll want to add in the costs of any improvements to your basis as well.

Based on your home's assessed value, annual real estate taxes are tax deductible. Your mortgage interest statement may list the amount of taxes you paid. If it doesn't, review canceled checks to figure out how much you've paid.

YOUR E-MAIL ALERTS Internal Revenue Service (IRS) Mortgages Financial and Business Services Loan Markets or Create your own Manage alerts | What is this?

If you own a cooperative, your coop will give you a statement of the taxes paid on your behalf that you can deduct. Keep in mind that water, sewer and tax collection fees charged by your local government cannot be written off. And assessments for a new street or sewer system likewise aren't deductible, although they add to the basis of your home.

When you're buying or selling real estate, the buyer and seller typically split the tax bill. You can only deduct the tax that you paid, not the amount due in that year.

Gerri Willis is the personal finance editor for CNN Business News. Willis also is co-host of CNNfn's The FlipSide, weekdays from 11 a.m. to 12:30 p.m. (ET). E-mail comments to 5tips@cnnfn.com.