Analyzing Difficulty of Machine Comprehension focusing on Examples Answerable by BERT without Question
-
- LI Hongyu
- University of Tsukuba
-
- CHEN Tengyang
- University of Tsukuba
-
- UTSURO Takehito
- University of Tsukuba
-
- KAWADA Yasuhide
- Logworks Co., Ltd.
Bibliographic Information
- Other Title
-
- 質問なし読解事例に対するBERTの回答可能性に着目した機械読解難易度分析
Abstract
<p>In the field of machine comprehension (MC), the task of an MC model is to predict the answer (A) from a question (Q) and related context (C) of the question. However, in this paper, it is discovered that there exist examples that can be correctly answered by an MC model BERT where only the context of the example is given without the question being given, which means that the difficulty of examples of machine comprehension vary. Based on this finding, this paper proposes a method based on BERT which splits the training examples of the MC dataset SQuAD1.1 into “easy to answer” and “hard to answer” ones. Experimental evaluation results of comparing the two models, one of which is trained with the “easy to answer” examples only, while the other of which is trained with the “hard to answer” examples only, show that the latter outperforms the former.</p>
Journal
-
- Proceedings of the Annual Conference of JSAI
-
Proceedings of the Annual Conference of JSAI JSAI2020 (0), 1E5GS903-1E5GS903, 2020
The Japanese Society for Artificial Intelligence