‘So you’re literally getting the opinion [of] people so f****** stupid they answer polls,’ Rogan said