Johnson was joking about her history of dating musicians when the “Cruel Summer” singer suddenly joined her Saturday night.