The goal is to find a balance between a region that allows you to maintain your financial security and one that doesn't compromise your quality of life.
As all too many retirees have learned, relocating to save money can be a jolting experience if it takes them far from family and friends. Factor in the cost of traveling to see loved ones if you move to a different part of the country. You don't want to be so far away that it becomes cost prohibitive to maintain your most important relationships.
Don't pick up and move without giving a new town a trial run. Rent a place for a few months to make sure you get a true sense of what the community is like. If you have your eye on an area that's very popular in the winter, spend some time there in summer. Maybe you will love the quiet - or maybe you'll find it depressing when the high-season action packs up and goes home.
While change and new adventures can be invigorating, make sure wherever you move lets you continue to pursue your favorite hobbies or pastimes. If you are a big opera and symphony fan, moving to the boonies might not be a good fit.
Make sure you'll have close access to good health care - especially in specialties of particular concern to older people, such as cancer, heart disease and general geriatric care.
Also carefully consider the financial pros and cons of the region you're considering. Look for a well-diversified economy - that means the area is more likely to be able to weather setbacks to a particular industry. In a one-company or one-industry town, your home's value can take a big hit if that company or industry hits a hard patch or decides to relocate. A protracted downslide in a region (or state) can eventually lead to cutbacks in basic public services, from a reduction in bus routes to a smaller police force.
You get the idea. New can be great. Just make sure that what's new is also what you love to do, or have nearby.