Home  |  Centre for Mathematical Sciences  |  LTH  |  LU
Title: Regularizing Mono- and Bi-Word Models for Word Alignment
Full text: PDF
Authors: Schoenemann, Thomas
Year: 2011
Document Type:Conference Paper
Conference: 5th International Joint Conference on Natural Language Processing
Status: Published
Refereed: Yes
BibTeX item:BibTeX
Abstract: Conditional probabilistic models for word alignment are popular due to the elegant way of handling them in the training stage. However, they have weaknesses such as garbage collection and scale poorly beyond single word based models (DeNero et al., 2006): not all parameters should actually be used. To alleviate the problem, in this paper we explore regularity terms that penalize the used parameters. They share the advantages of the standard training in that iterative schemes decompose over the sentence pairs. We explore the models IBM-1 and HMM, then generalize to models we term Bi-word models, where each target word can be aligned to up to two source words. We give two optimization strategies for the arising tasks, using EM and projected gradient descent. While both are well-known, to our knowledge they have never been compared experimentally for the task of word alignment. As a side-effect, we show that, against common belief, for parametric HMMs the M-step is not solved by renormalizing expectations. We demonstrate that the regularity terms improve on the f-measures of the standard HMMs and that they improve translation quality.




Questions: webmaster
Last updated: 2013-06-04

Centre for Mathematical Sciences, Box 118, SE-22100, Lund. Phone: 046-222 00 00