振舞複製による敵対的模倣学習高速化に関する考察

DOI Web Site

書誌事項

タイトル別名
  • A Study On Accelerating Adversarial Imitation Learning By Behavioral Cloning

説明

<p>Imitation learning is a popular method to obtain policies on autonomous robots given expert demonstrations. Recently, adversarial imitation learning methods, such as generative adversarial imitation learning (GAIL), have achieved great successes even on complex continuous control tasks. However, GAIL as well as its variants require a huge amount of environment interactions that often take impractically long time for training the robot. An intuitive way to reduce the number of interactions is initializing a policy by behavioral cloning (BC) before performing GAIL as pointed out in [1]. However, Sasaki et al reports that the BC initialization does not lead to reduce the number of interactions at all, rather significantly harms the imitation results. In this paper, we further analyze the BC initialization to figure out why the results are opposed to the intuition. Experimental results show that one of the cause of failure due to the BC initialization is that BC vanishes gradients of objective functions for the adversarial imitation learning algorithms, even though the objective differs from that of BC.</p>

収録刊行物

キーワード

詳細情報 詳細情報について

問題の指摘

ページトップへ