Scheduling of Damping in Natural Gradient Method

DOI

Bibliographic Information

Other Title
  • 自然勾配学習法における Damping 項のスケジューリング

Abstract

<p>In recent years, second-order optimization with a fast convergence rate has been used in deep learning owing to fast approximation methods for natural gradient methods. Second-order optimization requires the inverse computation of the information matrix, which generally degenerates in the deep learning problem. Therefore, as a heuristic, a damping method adds a unit matrix multiplied by a constant. This study proposed a method for scheduling damping motivated by the Levenberg-Marquardt method for determining damping and investigated its effectiveness.</p>

Journal

Keywords

Details 詳細情報について

Report a problem

Back to top