The former royal has had to halt horseback riding in a bid to avoid being photographed, a source said, ending another hobby after he handed over his gun licence last year