Gardeners' World star Adam Frost opened up about the bittersweet experience of moving house, nine months after relocating with his wife and children