Controllable Japanese Temporal Inference Dataset

DOI

Bibliographic Information

Other Title
  • 制御可能な日本語時間推論データセットの構築

Abstract

<p>Natural Language Inference (NLI) tasks that require temporal inference remain challenging for pre-trained language models (LMs). Although various datasets have been created for this task, they primarily focus on English and do not address the need for resources in other languages. In this paper, we present a Japanese NLI benchmark for temporal inference. To begin the data annotation process, we create inference templates consisting of various inference patterns based on the formal semantics test suites. We then automatically generate diverse NLI examples by assigning nouns, verbs, and temporal expressions to the templates using the Japanese case frame dictionary. We evaluate the generalization capacities of monolingual/multilingual LMs by using controlled splits of our dataset. Our findings demonstrate that LMs struggle with handling specific linguistic phenomena such as habituality.</p>

Journal

Keywords

Details 詳細情報について

Report a problem

Back to top