Make the NPCs their own houses, make sure there are no spaces in your backwall (you do have it, right?), if nothing helps, make lava moats and/or set bricks down your doors