Learning Parameters of Japanease Morphological Analyzer based-on Hidden Markov Model
- Other Title
Search this article
Morphological analysis is the first step toward the analysis of Japanese texts and one of the most important processes. So far, we have been developing the Japanease morphological analyzer JUMAN as a public-domain system. In JUMAN, ambiguities of morphological analysis are reduced by means of costs manually attached to the connectivity rules and words. The performance of JUMAN largely depends on those manually attached costs, while at present JUMAN has no facility to optimize the costs. This paper proposes a method for optimizing the costs (i.e. parameters) to be attached to the connectivity rules and words. The proposed method is based on hidden Markov model, which has proved effective in parameter estimation of English part-of-speech tagging. The result of experiments shows that the proposed optimization method improves the manually attached parameters.
- IPSJ SIG Notes
IPSJ SIG Notes 108 13-19, 1995-07-20
Information Processing Society of Japan (IPSJ)