書誌事項
- タイトル別名
-
- Evaluating Text Summarization Using Multiple Correct Answer Summaries
- テキスト ヨウヤク ノ フクスウ ノ セイカイ ニ モトヅイタ ヒョウカ
この論文をさがす
説明
We proposed an evaluation method based on multiple correct answer summaries. Conventional evaluation methods had reliability problem due to adopting single model answer while multiple correct answer summaries may exist from various points of view. We aimed to increase the reliability of automatic evaluation, and focused on an evaluation method using multiple answer summaries. In our method, we introduced linear combinations of answer summaries, all denoted by vectors, and calculated its maximum value of the scalar product for the answers and the target summary. To verify the reliability of our method, 7 people created summaries for 4 newspaper articles in NTCIR-2 summarization test collection data. However, low agreement among these answer summaries showed these data inadequate to be used as answers for the evaluation method. These summaries showed some tendency of keeping the text configurations due to anaphoric relations and sentence cohesions. Those findings will be valuable in creating model summaries. To verify the feasibility of the evaluation method, some automatic methods were evaluated using the multiple correct summaries. Most feasible method was varied according to each correct summary. The result has proved our presupposed theory, that multiple correct answers were necessary to sufficiently evaluate the target summary data.
収録刊行物
-
- 自然言語処理
-
自然言語処理 9 (4), 33-53, 2002
一般社団法人 言語処理学会
- Tweet
詳細情報 詳細情報について
-
- CRID
- 1390282679450677248
-
- NII論文ID
- 10010197956
-
- NII書誌ID
- AN10472659
-
- ISSN
- 21858314
- 13407619
-
- NDL書誌ID
- 6674484
-
- 本文言語コード
- ja
-
- データソース種別
-
- JaLC
- NDLサーチ
- Crossref
- CiNii Articles
-
- 抄録ライセンスフラグ
- 使用不可
