The ABC franchise was once a leader in the unscripted realm, but its recent struggles reflect a framework that is increasingly out of step with modern reality TV trends.