Victoria Coren Mitchell opened up about the unexpected feedback she recieved about her BBC game show, Only Connect