A mild microclimate, an exciting locavore food scene and a distinct Mediterranean look make this seaside town a must-visit destination, says Hayley Spencer