-
- Sabrina J. Mielke
- Department of Computer Science, Johns Hopkins University, USA
-
- Arthur Szlam
- Facebook AI Research, USA. aszlam@fb.com
-
- Emily Dinan
- Facebook AI Research, USA. edinan@fb.com
-
- Y-Lan Boureau
- Facebook AI Research, USA. ylan@fb.com
書誌事項
- 公開日
- 2022
- 権利情報
-
- https://creativecommons.org/licenses/by/4.0/
- DOI
-
- 10.1162/tacl_a_00494
- 公開者
- MIT Press
説明
<jats:title>Abstract</jats:title> <jats:p>While improving neural dialogue agents’ factual accuracy is the object of much research, another important aspect of communication, less studied in the setting of neural dialogue, is transparency about ignorance. In this work, we analyze to what extent state-of-the-art chit-chat models are linguistically calibrated in the sense that their verbalized expression of doubt (or confidence) matches the likelihood that the model’s responses are factually incorrect (or correct). We find that these models are poorly calibrated, yet we show that likelihood of correctness can accurately be predicted. By incorporating such metacognitive features into the training of a controllable generation model, we obtain a dialogue agent with greatly improved linguistic calibration.</jats:p>
収録刊行物
-
- Transactions of the Association for Computational Linguistics
-
Transactions of the Association for Computational Linguistics 10 857-872, 2022
MIT Press