Scrapping the VAT refund scheme for international visitors in 2021 left the UK ‘less competitive’, the Tories said.