Stephen Schwartz says he will not appear at the arts venue as it "has now become an ideological statement".