Exactly! Just tell him you are expecting X to pic it up on this day when you knew, then he probably wouldn’t pester you so much.