‘It was an ‘if you build it, they will come’ mentality that failed,’ one source close to the White House said