How do synthetic deduction corpora enhance language models?

DOI

Bibliographic Information

Other Title
  • 人工演繹推論コーパスによる学習は言語モデルをどのように強化するか？

Abstract

<p>We study synthetic corpus-based approaches for language models (LMs) to acquire logical deductive reasoning ability. The previous studies trained LMs on synthetically generated examples of deductive reasoning, which have been effective to an extent. However, it has not yet been studied on what aspect of deductive reasoning ability deduction corpora have enhanced LMs. This investigation is essential to discuss the future directions of deductive reasoning. We investigate this by generating and using a comprehensive set of ``ablation corpora'', where one corpus emphasizes a specific aspect different from those emphasized by the other corpora. Finally, on the basis of these results, we discuss the future directions for applying deduction corpora or other approaches for each aspect.</p>

Journal

Keywords

Details 詳細情報について

Report a problem

Back to top