SCIEUR Damien

< Back to ILB Patrimony
Affiliations
  • 2017 - 2018
    Département d'Informatique de l'Ecole Normale Supérieure
  • 2017 - 2018
    Ecole normale supérieure Paris
  • 2017 - 2018
    Communauté d'universités et établissements Université de Recherche Paris Sciences et Lettres
  • 2017 - 2018
    Sciences mathematiques de paris centre
  • 2017 - 2018
    Apprentissage statistique et parcimonie
  • 2021
  • 2018
  • Super-Acceleration with Cyclical Step-sizes.

    Baptiste GOUJAUD, Damien SCIEUR, Aymeric DIEULEVEUT, Adrien TAYLOR, Fabian PEDREGOSA
    2021
    Cyclical step-sizes are becoming increasingly popular in the optimization of deep learning problems. Motivated by recent observations on the spectral gaps of Hessians in machine learning, we show that these step-size schedules offer a simple way to exploit them. More precisely, we develop a convergence rate analysis for quadratic objectives that provides optimal parameters and shows that cyclical learning rates can improve upon traditional lower complexity bounds. We further propose a systematic approach to design optimal first order methods for quadratic minimization with a given spectral structure. Finally, we provide a local convergence rate analysis beyond quadratic minimization for the proposed methods and illustrate our findings through benchmarks on least squares and logistic regression problems.
Affiliations are detected from the signatures of publications identified in scanR. An author can therefore appear to be affiliated with several structures or supervisors according to these signatures. The dates displayed correspond only to the dates of the publications found. For more information, see https://scanr.enseignementsup-recherche.gouv.fr