Offline Model-Based Imitation Learning with Entropy Regularization of Model and Policy

DOI
  • UCHIBE Eiji
    Advanced Telecommunications Research Institute International

Bibliographic Information

Other Title
  • 方策とモデルのエントロピ正則を導入したオフラインモデルベース模倣学習

Abstract

<p>Model-Based Entropy-Regularized Imitation Learning (MB-ERIL) is an online model-based generative adversarial imitation learning method that introduces entropy regularization of policy and state transition model. Online-MB-ERIL learns the policy and model from expert data, learner's data, and generated data. Costly interactions with an actual environment are needed to obtain the first two datasets, while the policy and model quickly generate the last one. This report discusses an offline learning setting without using the second data obtained from the interaction between the policy and the actual environment. Next, we propose Offline-MB-ERIL, which introduces the idea of Positive and Unlabeled data learning. Given sub-optimal data, Offline-MB-ERIL can recover policy and model efficiently using them as unlabeled data. Through a vision-based arm-reaching task, we show that Offline-MB-ERIL can better use suboptimal data than Online-MB-ERIL.</p>

Journal

Keywords

Details 詳細情報について

Report a problem

Back to top