Input noise injection for supervised machine learning, with applications on genomic and image data.

Summary Overlearning is a general problem that affects statistical learning algorithms in different ways and has been approached in different ways in the literature. We first illustrate a real case of this problem in the context of a collaborative work aiming at predicting the response of rheumatoid arthritis patients to anti-inflammatory treatments. We then focus on the Noise Injection method in data in its generality as a regularization method. We give an overview of this method, its applications, insights, algorithms and some theoretical elements in the context of supervised learning. We then focus on the dropout method introduced in the context of deep learning and construct a new approximation allowing a new interpretation of this method in a general framework. We complement this study with experiments on simulations and real data. We then present a generalization of the noise injection method inspired by the noise inherent to certain types of data, which also allows variable selection. We present a new stochastic algorithm for this method, study its regularization properties and apply it to the context of single cell RNA sequencing. Finally, we present another generalization of the Noise Injection method where the introduced noise follows a structure that is adaptively inferred from the model parameters, as the covariance of the activations of the units to which it is applied. We study the theoretical properties of this new method called ASNI for linear models and multilayer neural networks. Finally, we show that ASNI improves the generalization performance of predictive models while improving the resulting representations.