Venus Williams' memorable US Open doubles run came to an end with a quarter-final loss at Flushing Meadows.