Despite progress, councillor Jamie Hirst said discrimination was getting worse in recent years as people targeted his family.