Monty Don opened up about the backlash he received over a comment he made on BBC Gardeners' World, just days before announcing that he is leaving the show