Employers say many of the departing workers had valuable sewing skills that are hard to replace quickly, while locals are often unwilling to take the low-paying jobs.