One of England's most iconic grounds was reduced to rubble, which then helped lay the foundations for a little-known monument elsewhere in London