Anticipation Captioning with Commonsense Knowledge

DOI Web Site

この論文をさがす

抄録

<p>In this review, we introduce a novel image captioning task, called Anticipation Captioning, which generates a caption for an unseen image given a sparsely temporally-ordered set of images. Our task emulates the human capacity to reason about the future based on a sparse collection of visual cues acquired over time. To address this novel challenge, we introduce a model, namely A-CAP, that predicts the caption by incorporating commonsense knowledge into a pre-trained vision-language model. Our method outperforms image captioning methods and provides a solid baseline for anticipation captioning task, as shown in both qualitative and quantitative evaluations on a customized visual storytelling dataset. We also discuss the potential applications, challenges, and future directions of this novel task.</p>

収録刊行物

  • 日本画像学会誌

    日本画像学会誌 62 (6), 588-598, 2023-12-10

    一般社団法人 日本画像学会

キーワード

詳細情報 詳細情報について

問題の指摘

ページトップへ