A PAC-Bayesian analysis of domain adaptation and its specialization to linear classifiers.

Summary In this paper, we focus on the domain adaptation (DA) problem corresponding to the case where the training and test data are from different distributions. We propose a PAC-Bayesian analysis of this problem in the context of binary classification without supervised information on the test data. The PAC-Bayesian theory provides theoretical guarantees on the risk of a majority vote on a set of hypotheses. Our contribution to the DA framework relies on a new measure of divergence between distributions based on a notion of expectation of disagreement between hypotheses. This measure allows us to derive a first PAC-Bayesian bound for the stochastic Gibbs classifier. This bound has the advantage of being directly optimizable for any hypothesis space and we give an illustration in the case of linear classifiers. The algorithm proposed in this context shows interesting results on a toy problem as well as on a common opinion analysis task. These results open new perspectives for understanding the domain adaptation problem thanks to the tools offered by the PAC-Bayesian theory.