Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations
Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations
Sparse language vectors from linguistic typology databases and learned embeddings from tasks like multilingual machine translation have been investigated in isolation, without analysing how they could benefit from each other's language characterisation. We propose to fuse both views using singular vector canonical correlation analysis and study what kind of information is induced from each source. By inferring typological features and language phylogenies, we observe that our representations embed typology and strengthen correlations with language relationships. We then take advantage of our multi-view language vector space for multilingual machine translation, where we achieve competitive overall translation accuracy in tasks that require information about language similarities, such as language clustering and ranking candidates for multilingual transfer. With our method, which is also released as a tool, we can easily project and assess new languages without expensive retraining of massive multilingual or ranking models, which are major disadvantages of related approaches.
Comment: Accepted at EMNLP 2020. Camera-ready version
Microsoft Academic Graph classification: Machine translation business.industry Computer science computer.software_genre Bridging (programming) Linguistic typology Artificial intelligence business Cluster analysis computer Natural language processing
FOS: Computer and information sciences, Computer Science - Computation and Language, Computation and Language (cs.CL)
FOS: Computer and information sciences, Computer Science - Computation and Language, Computation and Language (cs.CL)
Microsoft Academic Graph classification: Machine translation business.industry Computer science computer.software_genre Bridging (programming) Linguistic typology Artificial intelligence business Cluster analysis computer Natural language processing
4615
Antonios Anastasopoulos. 2019. A note on evaluating multilingual benchmarks.
Johannes Bjerva and Isabelle Augenstein. 2018a. From phonology to syntax: Unsupervised linguistic typology at different levels with language embeddings. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 907-916, New Orleans, Louisiana. Association for Computational Linguistics.
Johannes Bjerva and Isabelle Augenstein. 2018b. Tracking typological traits of uralic languages in distributed language representations. In Proceedings of the Fourth International Workshop on Computational Linguistics of Uralic Languages, pages 76-86, Helsinki, Finland. Association for Computational Linguistics.
Johannes Bjerva, Yova Kementchedjhieva, Ryan Cotterell, and Isabelle Augenstein. 2019a. A probabilistic generative model of linguistic typology. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1529-1540, Minneapolis, Minnesota. Association for Computational Linguistics. [OpenAIRE]
Johannes Bjerva, Robert stling, Maria Han Veiga, Jrg Tiedemann, and Isabelle Augenstein. 2019b. What do language representations really represent? Computational Linguistics, 45(2):381-389. [OpenAIRE]
Mauro Cettolo, Christian Girardi, and Marcello Federico. 2012. WIT3: Web inventory of transcribed and translated talks. In Proceedings of the 16th Conference of the European Association for Machine Translation (EAMT), pages 261-268, Trento, Italy.
Bernard Comrie. 1989. Language universals and linguistic typology: Syntax and morphology. University of Chicago press.
Paramveer S Dhillon, Dean P Foster, and Lyle H Ungar. 2015. Eigenwords: Spectral word embeddings. The Journal of Machine Learning Research, 16(1):3035- 3078.
Matthew S. Dryer and Martin Haspelmath, editors. 2013. WALS Online. Max Planck Institute for Evolutionary Anthropology, Leipzig.
Susan T Dumais. 2004. Latent semantic analysis. Annual review of information science and technology, 38(1):188-230.
citations This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).10 popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.Top 10% influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).Average impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.Top 10% citations This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).10 popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.Top 10% influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).Average impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.Top 10%
Citations provided by BIP!Popularity provided by BIP!