"We don't want to be a shelter. We don't want to be program housing," one disgruntled resident told The Post.