[This article was first published on mlr-org, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)Want to share your content on R-bloggers? click here if you have a blog, or here if you don't. ScopeFitted model objects in R often carry more than they need for prediction. A glm object keeps the training data, the fitted values, the residuals, and the environment of its formula. That is fine for a single model. It adds up when resample() or tune() keep one model per resampling iteration with store_models = TRUE.The butcher package from the tidymodels ecosystem removes these components while keeping the model usable for prediction. This post shows how to apply it to a single learner and how to wrap it in a callback so that every model stored by mlr3 and mlr3tuning gets shrunk right after training.library(mlr3)library(mlr3learners)library(mlr3tuning)library(mlr3misc)library(butcher)We measure the size of an object by serializing it. This is the number of bytes the object needs on disk or when it is sent to a parallel worker.serialized_size = function(x) { round(length(serialize(x, connection = NULL)) / 1024^2, digits = 2L)}Butcher a single modelWe train a logistic regression on the spam task with 4601 emails and 57 features.task = tsk("spam")learner = lrn("classif.log_reg", predict_type = "prob")learner$train(task)serialized_size(learner$model)[1] 4.44The model needs about 4.4 MB. The function weigh() lists the components of the model object sorted by size in MB.weigh(learner$model)# A tibble: 101 × 2 object size 1 qr.qr 2.16 2 residuals 0.333 3 fitted.values 0.333 4 linear.predictors 0.333 5 weights 0.333 6 prior.weights 0.333 7 y 0.333 8 effects 0.0773 9 family.variance 0.044010 family.dev.resids 0.0440# ℹ 91 more rowsThe QR decomposition is the largest component. The residuals, fitted values, linear predictors, weights, and the response each take another 0.33 MB. Prediction needs only a fraction of this.We predict on the task, butcher the model, and write it back into the learner.prediction = learner$predict(task)learner$model = butcher(learner$model)serialized_size(learner$model)[1] 2.34The model shrank to roughly half its size. The predictions of the butchered model are identical to the original predictions.all.equal(prediction, learner$predict(task))[1] TRUEThe function butcher() only removes components that the predict() method of the model class does not use. It keeps the QR decomposition because predict.glm() needs it to compute standard errors.WarningA butchered model has lost information. Functions such as summary(), residuals(), or fitted() may fail or return wrong results on it. Only butcher models that you need for prediction.Resample callbackWe now want to shrink every model that resample() stores. Callbacks run code at specific stages of the resampling loop. We create a resample callback with mlr3::callback_resample(). The stage on_resample_end runs after the learner has been trained and has made its predictions. The context object gives access to the learner of the current iteration.callback = callback_resample("butcher", on_resample_end = function(callback, context) { context$learner$model = butcher::butcher(context$learner$model) })We call butcher() with the package prefix because the callback may run on a parallel worker on which butcher is not attached.We run a 5-fold cross-validation twice on the same splits, once without and once with the callback.resampling = rsmp("cv", folds = 5L)resampling$instantiate(task)rr = resample(task, learner, resampling, store_models = TRUE)rr_butchered = resample(task, learner, resampling, store_models = TRUE, callbacks = callback)We compare the size of the stored models.serialized_size(map(rr$learners, "model"))[1] 19.19serialized_size(map(rr_butchered$learners, "model"))[1] 10.4The five butchered models need about 9 MB less. The callback only touches the stored models, so the performance estimate does not change.rr$aggregate(msr("classif.auc"))classif.auc 0.971403 rr_butchered$aggregate(msr("classif.auc"))classif.auc 0.971403 The stored learners still predict new data.rr_butchered$learners[[1L]]$predict(task)── for 4601 observations: ────────────────────────────────────────────────────────────────────────── row_ids truth response prob.spam prob.nonspam 1 spam spam 0.62852865 3.714714e-01 2 spam spam 0.99139839 8.601607e-03 3 spam spam 0.99999999 5.408266e-09 --- --- --- --- --- 4599 nonspam nonspam 0.03097766 9.690223e-01 4600 nonspam nonspam 0.07923412 9.207659e-01 4601 nonspam nonspam 0.02782719 9.721728e-01benchmark() accepts the same callbacks argument.TuningTuning with store_models = TRUE stores one model per resampling iteration and hyperparameter configuration. Tuning instances take tuning callbacks, which can include the resample stages. We create one with mlr3tuning::callback_batch_tuning().callback_tuning = callback_batch_tuning("butcher", on_resample_end = function(callback, context) { context$learner$model = butcher::butcher(context$learner$model) })We tune the complexity parameter of a decision tree with 20 random configurations and a 3-fold cross-validation.learner_rpart = lrn("classif.rpart", cp = to_tune(1e-4, 1e-1, logscale = TRUE))instance = tune( tuner = tnr("random_search"), task = task, learner = learner_rpart, resampling = rsmp("cv", folds = 3L), term_evals = 20L, store_models = TRUE, callbacks = callback_tuning)The archive contains a benchmark result with the 60 butchered models.models = map(instance$archive$benchmark_result$resample_results$resample_result, function(rr) { map(rr$learners, "model")})serialized_size(models)[1] 2.68ConclusionThe butcher package removes data, calls, and environments that a model does not need for prediction. Wrapped in a callback, it shrinks every model that resample(), benchmark(), or tune() stores. The stored results take less memory and are faster to save to disk and to transfer from parallel workers. How much it saves depends on the model class. For model classes without a butcher method, butcher() returns the model unchanged. You can check which classes are supported in the butcher documentation. To leave a comment for the author, please follow the link and comment on their blog: mlr-org.R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.Continue reading: Shrink stored models with butcher