Once you know what you can afford for the loan, take another look at total costs for running the house.
At the very least, you'll have to cover taxes, insurance, maintenance and utilities. If you live more than an hour away, you might have to factor in the cost of a caretaker or property manager. Also, when you figure maintenance costs, get an idea from someone who lives in the area full-time about what it really takes to keep a house in good condition.
In beach and mountain areas, sand, salt, wind and ice can raise a home's maintenance costs substantially. A beach house may need to be painted every two or three years, and a mountain house may need annual deck repairs.
Whatever you do, when you figure the costs, inflate the numbers. If you're figuring on a worst-case scenario, you'll rarely run across a surprise.