Jermain Defoe's sudden exit as manager of Woking came as a surprise - and the National League side's director of football has now claimed it was prompted by his no-shows