publication . Conference object . Preprint . 2017

Unlabeled Data for Morphological Generation With Character-Based Sequence-to-Sequence Models

Katharina Kann; Hinrich Schütze;
Open Access
  • Published: 17 May 2017
  • Publisher: Association for Computational Linguistics
Abstract
We present a semi-supervised way of training a character-based encoder-decoder recurrent neural network for morphological reinflection, the task of generating one inflected word form from another. This is achieved by using unlabeled tokens or random strings as training data for an autoencoding task, adapting a network for morphological reinflection, and performing multi-task training. We thus use limited labeled data more effectively, obtaining up to 9.9% improvement over state-of-the-art baselines for 8 different languages.
Persistent Identifiers
Subjects
ACM Computing Classification System: ComputingMethodologies_PATTERNRECOGNITION
free text keywords: Computer Science - Computation and Language, Training set, Recurrent neural network, Pattern recognition, Computer science, Labeled data, Artificial intelligence, business.industry, business
25 references, page 1 of 2

Roee Aharoni, Yoav Goldberg, and Yonatan Belinkov. 2016. Improving sequence to sequence learning for morphological inflection generation: The BIU-MIT systems for the SIGMORPHON 2016 shared task for morphological reinflection. In SIGMORPHON.

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In ICLR.

Yong Cheng, Wei Xu, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. 2016. Semisupervised learning for neural machine translation. arXiv preprint arXiv:1606.04596 .

Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014a. On the properties of neural machine translation: Encoder-decoder approaches. In SSST.

Kyunghyun Cho, Bart Van Merrie¨nboer, C¸alar Gu¨lc¸ehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014b. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In EMNLP.

Ryan Cotterell, Christo Kirov, John Sylak-Glassman, David Yarowsky, Jason Eisner, and Mans Hulden. 2016. The SIGMORPHON 2016 shared taskmorphological reinflection. In SIGMORPHON.

Mathias Creutz and Krista Lagus. 2007. Unsupervised models for morpheme segmentation and morphology learning. TSLP 4(1):3.

Markus Dreyer and Jason Eisner. 2009. models over multiple strings. In EMNLP.

Greg Durrett and John DeNero. 2013. Supervised learning of complete morphological paradigms. In NAACL.

Manaal Faruqui, Yulia Tsvetkov, Graham Neubig, and Chris Dyer. 2016. Morphological inflection generation using character sequence to sequence learning. In NAACL.

John Goldsmith. 2001. Unsupervised learning of the morphology of a natural language. Computational linguistics 27(2):153-198.

Thanh-Le Ha, Jan Niehues, and Alexander Waibel. 2016. Toward multilingual neural machine translation with universal encoder and decoder. arXiv preprint arXiv:1611.04798 . [OpenAIRE]

Katharina Kann, Ryan Cotterell, and Hinrich Schu¨tze. 2017. One-shot neural cross-lingual transfer for paradig completion. In ACL.

Katharina Kann and Hinrich Schu¨tze. 2016. MED: The LMU system for the SIGMORPHON 2016 shared task on morphological reinflection. In ACL.

Garrett Nicolai, Colin Cherry, and Grzegorz Kondrak. 2015. Inflection generation as discriminative string transduction. In NAACL.

25 references, page 1 of 2
Abstract
We present a semi-supervised way of training a character-based encoder-decoder recurrent neural network for morphological reinflection, the task of generating one inflected word form from another. This is achieved by using unlabeled tokens or random strings as training data for an autoencoding task, adapting a network for morphological reinflection, and performing multi-task training. We thus use limited labeled data more effectively, obtaining up to 9.9% improvement over state-of-the-art baselines for 8 different languages.
Persistent Identifiers
Subjects
ACM Computing Classification System: ComputingMethodologies_PATTERNRECOGNITION
free text keywords: Computer Science - Computation and Language, Training set, Recurrent neural network, Pattern recognition, Computer science, Labeled data, Artificial intelligence, business.industry, business
25 references, page 1 of 2

Roee Aharoni, Yoav Goldberg, and Yonatan Belinkov. 2016. Improving sequence to sequence learning for morphological inflection generation: The BIU-MIT systems for the SIGMORPHON 2016 shared task for morphological reinflection. In SIGMORPHON.

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In ICLR.

Yong Cheng, Wei Xu, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. 2016. Semisupervised learning for neural machine translation. arXiv preprint arXiv:1606.04596 .

Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014a. On the properties of neural machine translation: Encoder-decoder approaches. In SSST.

Kyunghyun Cho, Bart Van Merrie¨nboer, C¸alar Gu¨lc¸ehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014b. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In EMNLP.

Ryan Cotterell, Christo Kirov, John Sylak-Glassman, David Yarowsky, Jason Eisner, and Mans Hulden. 2016. The SIGMORPHON 2016 shared taskmorphological reinflection. In SIGMORPHON.

Mathias Creutz and Krista Lagus. 2007. Unsupervised models for morpheme segmentation and morphology learning. TSLP 4(1):3.

Markus Dreyer and Jason Eisner. 2009. models over multiple strings. In EMNLP.

Greg Durrett and John DeNero. 2013. Supervised learning of complete morphological paradigms. In NAACL.

Manaal Faruqui, Yulia Tsvetkov, Graham Neubig, and Chris Dyer. 2016. Morphological inflection generation using character sequence to sequence learning. In NAACL.

John Goldsmith. 2001. Unsupervised learning of the morphology of a natural language. Computational linguistics 27(2):153-198.

Thanh-Le Ha, Jan Niehues, and Alexander Waibel. 2016. Toward multilingual neural machine translation with universal encoder and decoder. arXiv preprint arXiv:1611.04798 . [OpenAIRE]

Katharina Kann, Ryan Cotterell, and Hinrich Schu¨tze. 2017. One-shot neural cross-lingual transfer for paradig completion. In ACL.

Katharina Kann and Hinrich Schu¨tze. 2016. MED: The LMU system for the SIGMORPHON 2016 shared task on morphological reinflection. In ACL.

Garrett Nicolai, Colin Cherry, and Grzegorz Kondrak. 2015. Inflection generation as discriminative string transduction. In NAACL.

25 references, page 1 of 2
Any information missing or wrong?Report an Issue