When forming prospective judgments of distraction, people do not have direct metacognitive access to the cognitive effects of task-irrelevant sounds that determine auditory distraction. Instead, they rely on a fluency heuristic, using the subjective ease or difficulty with which a sound is processed as a metacognitive cue. Empirically, this conclusion is supported by metacognitive illusions, with judgments of distraction failing to align with objective distraction. Here we address why and when the fluency heuristic leads to incorrect or correct judgments of distraction. Specifically, we test the hypothesis that fluency reflects processing at multiple processing stages, not all of which are relevant for auditory distraction in the task at hand. Across two experiments, we manipulated the similarity of words within word sequences at two stages of speech processing. In Experiment 1, conceptual similarity was manipulated to influence fluency at the conceptual processing stage. In Experiment 2, perceptual similarity was manipulated to influence fluency at the perceptual processing stage. In line with the fluency heuristic, participants predicted sequences of similar words—whether conceptually or perceptually similar—to be less distracting than sequences of dissimilar words. Objective distraction in serial recall was unaffected by conceptual similarity but was affected by perceptual similarity. These results suggest that fluency reflects processing at multiple processing stages and thereby influences judgments of distraction, whereas objective distraction depends on whether the corresponding processing stage determines auditory distraction in the task at hand. Metacognitive illusions arise when fluency is generated at processing stages that do not determine auditory distraction.