The Foreign Secretary accused previous governments of having had a tendency to ‘sweep under the carpet’ the issue of slavery