Advocacy groups have warned that the removal of former allies and their families could endanger their lives.