Optimal selection of localized basis models in heteroskedastic regression.

Summary The notion of localized basis is a fruitful concept in the theory of linear approximation, which unites histograms, piecewise polynomials and compactly supported wavelets, among others. We consider the problem of selecting the number of non-zero coefficients in a localized basis development, for the non-parametric estimation of a regression function, with random design and hetroscedastic noise. We then prove by setting up oracle inequalities the asymptotic optimality of the slope heuristic and of a V-fold penalization strategy. We also show that the classical V-fold cross-validation procedure is asymptotically suboptimal in the sense that it finds in...ni the oracle constructed from a fraction of the initial data equal to (V-1)/V. We conclude the paper with a simulatory study on wavelet models, where we note a noticeable difference between the asymptotic results of the theoretical framework and the finite distance practice. Indeed, the V-fold cross-validation and its variant by penalization give in our experiments comparable results while the asymptotic superiority of the penalization is proven.