The complaints against him include allegations that he made indecent, double-entendre remarks that seemed to be directed at an actor during his speech