Robust response transformations for generalized additive models via additivity and variance stabilisation

The AVAS (Additivity And Variance Stabilization) algorithm of Tibshirani provides a non-parametric transformation of the response in a linear model to approximately constant variance. It is thus a generalization of the much used Box-Cox transformation. However, AVAS is not robust. Outliers can have a major effect on the estimated transformations both of the response and of the transformed explanatory variables in the Generalized Additive Model (GAM).We describe and illustrate robust methods for the non-parametric transformation of the response and for estimation of the terms in the model and report the results of a simulation study comparing our robust procedure with AVAS. We illustrate the efficacy of our procedure through a simulation study and the analysis of real data.