Maybe both are irresponsible. But they are both responsible for the upkeep of the child now it's here, and currently only one of them is actually doing it. As you can't go back in time, sorry dude, suck it up and deal with it.