Character-Based Thai Word Segmentation with Multiple Attentions

DOI DOI Web Site 参考文献28件 オープンアクセス

この論文をさがす

説明

<p>Character-based word segmentation models have been extensively applied to Asian languages, including Thai, owing to their promising performance. These models estimate the word boundaries from a character sequence; however, a Thai character unit in a sequence has no inherent meaning, in contrast with word, subword, and character cluster units that represent more meaningful linguistic information. In this paper, we propose a Thai word segmentation model that uses various types of information, including words, subwords, and character clusters, from a character sequence. Our model applies multiple attentions to refine segmentation inferences by estimating the significant relationships among characters and various unit types. We evaluated our model on three Thai datasets, and the experimental results show that our model outperforms other Thai word segmentation models, demonstrating the validity of using character clusters over subword units. A case study on sample Thai text supported these results. Thus, according to our analysis, particularly the case study, our model can segment Thai text accurately, while other existing models yield incorrect results that violate the Thai writing system.</p>

収録刊行物

  • 自然言語処理

    自然言語処理 30 (2), 372-400, 2023

    一般社団法人　言語処理学会

参考文献 (28)*注記

もっと見る

キーワード

詳細情報 詳細情報について

問題の指摘

ページトップへ