Lord Kinnock says he understands it would not be easy but ‘if we really want sustained economic growth, we have to be part of a body of the single market with our nearest neighbours’