Prince Andrew should give evidence to US authorities, a government minister has said, as anger grows after it emerged he had been paying "peppercorn rent" for two decades.