The Prime Minister said it was unfair for football fans to be singled out when supporters of other sports were able to drink while watching their teams