Gladiators star Jamie Bigg, known as Giant, announced his departure from the popular BBC show earlier this week, claiming the decision was out of his control