Sunday Brunch was thrown into chaos after two guests had to suddenly leave the set just minutes into their live interview