Model-agnostic prediction intervals in Python and R: does nnetsauce’s `QuantileRegressor` hold up?

Wait 5 sec.

[This article was first published on T. Moudiki's Webpage - R, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.Model-agnostic prediction intervals in Python and R: does nnetsauce’s QuantileRegressor hold up?Point predictions tell you what a model thinks will happen. They don’t tell you howmuch to trust that number.nnetsauce’s QuantileRegressor classtakes a different approach to this problem than most prediction-interval libraries:instead of shipping one interval-producing algorithm, it takes any object with.fit()/.predict() — linear model, SVR, random forest, whatever you already have —and turns it into a full quantile machine by optimizing an offset around its pointpredictions to minimize the pinball (quantile) loss. Five different “scoring”strategies control how that offset is computed: predictions, residuals,conformal, studentized, conformal-studentized.The library, in Python and RThe same class is available in R, viannetsauce_r — and it’s worth notingup front that it isn’t a separate reimplementation. The R function is a thinreticulate wrapper that calls the identical Python object under the hood.There is no separate R implementation to audit; auditing the Python source isauditing the R behavior.Python (this is real, runnable code — see the cell below)from nnetsauce.quantile.quantileregression import QuantileRegressorobj = QuantileRegressor( obj=BayesianRidge(), # any sklearn-compatible regressor level=95, # target coverage, in % scoring="residuals", # "predictions" | "residuals" | "conformal" | # "studentized" | "conformal-studentized")obj.fit(X_train, y_train)result = obj.predict(X_test, return_pi=True)# result.mean, result.lower, result.median, result.upperR (calls into the exact same Python class via reticulate)library(datasets)X