Manchester City's Premier League rivals Arsenal crashed out of the FA Cup on Saturday night after a 2-1 defeat to Championship club Southampton.