Earlier this month, the Duke of Sussex's invitation to Buckingham Palace was “withdrawn at the last moment” after he “formally” accepted it.