Unsupervised Word Alignment Using Frequency Constraint in Posterior Regularized EM
-
- Kamigaito Hidetaka
- Department of Computational Intelligence and Systems Science, Interdisciplinary Graduate School of Science and Engineering, Tokyo Institute of Technology
-
- Watanabe Taro
- Google Inc.
-
- Takamura Hiroya
- Tokyo Institute of Technology, Precision and Intelligence Laboratory
-
- Okumura Manabu
- Tokyo Institute of Technology, Precision and Intelligence Laboratory
-
- Sumita Eiichiro
- National Institute of Information and Communication Technology
抄録
<p>Generative word alignment models, such as IBMModels, are restricted to one-to-many alignment, and cannot explicitly represent many-to-many relationships in bilingual texts. The problem is partially solved either by introducing heuristics or by agreement constraints such that two directional word alignments agree with each other. However, this constraint cannot take into account the grammatical difference of language pairs. In particular, function words are not trivial to align for grammatically different language pairs, such as Japanese and English. In this paper, we focus on the posterior regularization framework (Ganchev, Graca, Gillenwater, and Taskar 2010) that can force two directional word alignment models to agree with each other during training, and propose new constraints that can take into account the difference between function words and content words. We discriminate a function word and a content word using word frequency in the same way as done by Setiawan, Kan, and Li (2007). Experimental results show that our proposed constraints achieved better alignment qualities on the French-English Hansard task and the Japanese-English Kyoto free translation task (KFTT) measured by AER and F-measure. In translation evaluations, we achieved statistically significant gains in BLEU scores in the Japanese-English NTCIR10 task and Spanish-English WMT06 task.</p>
収録刊行物
-
- Information and Media Technologies
-
Information and Media Technologies 12 (0), 46-70, 2017
Information and Media Technologies 編集運営会議
- Tweet
キーワード
詳細情報 詳細情報について
-
- CRID
- 1390001205264407424
-
- NII論文ID
- 130005439846
-
- ISSN
- 18810896
-
- 本文言語コード
- en
-
- データソース種別
-
- JaLC
- CiNii Articles
-
- 抄録ライセンスフラグ
- 使用不可