Neural Machine Translation with CKY-based Convolutional Attention

Watanabe Taiki, Tamura Akihiro, Ninomiya Takashi, Bharata Adji Teguh

doi:10.5715/jnlp.26.207

Bibliographic Information

Other Title

CKY に基づく畳み込みアテンション構造を用いたニューラル機械翻訳
CKY ニモトズクタタミコミアテンションコウゾウオモチイタニューラルキカイホンヤク

Search this article

Abstract

<p>This paper proposes a new attention mechanism for neural machine translation (NMT) based on convolutional neural networks (CNNs), which is inspired by the CKY algorithm. The proposed attention represents every possible combination of source words (e.g., phrases and structures) through CNNs, which imitates the CKY table in the algorithm. NMT, incorporating the proposed attention, decodes a target sentence on the basis of the attention scores of the hidden states of CNNs. The proposed attention enables NMT to capture alignments from underlying structures of a source sentence without sentence parsing. The evaluations on the Asian Scientific Paper Excerpt Corpus (ASPEC) English-Japanese translation task show that the proposed attention gains 1.43 points in BLEU as compared to a conventional attention-based encoder decoder model. Furthermore, the proposed attention is at least comparable to, or better than, a conventional attention-based encoder decoder model on the FBIS Chinese-English translation task. </p>

Journal

Journal of Natural Language Processing

Journal of Natural Language Processing 26 (1), 207-230, 2019-03-15

The Association for Natural Language Processing

Keywords

Details 詳細情報について

Export