‘There’s a lot of things I’m passionate about, it’s not one of them… That’s someone else’s thing,’ says the Irish actor