Federal layoffs and an end to diversity initiatives have weakened a historically strong labor market for Black workers.