Blake Stephenson, the MP for Mid Bedfordshire, was speaking at a fringe event at the Conservative party conference when he made the remark