The building, which was once home to a 80,000-square-ft Marks & Spencer store, will be knocked down and an MP is worried the city centre could become a "derelict wasteland"