[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/torregrosa-2019-leveraging-rule-based-machine-translation/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/mtrans/) / Leveraging Rule-Based Machine Translation Knowledge for Under-Resourced Neural Machine Translation Models

# Leveraging Rule-Based Machine Translation Knowledge for Under-Resourced Neural Machine Translation Models

**Authors:** Daniel Torregrosa, Nivranshu Pasricha, Maraim Masoud, Bharathi Raja Chakravarthi, Juan Alonso, Noe Casas, Mihael Arcan · **Year:** 2019 · **Venue:** MT Summit XVII, Translator, Project and User Tracks **Link:** [aclanthology.org/W19-6725](https://aclanthology.org/W19-6725/) · **Licence:** [CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0/) — summary only

[**7** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/torregrosa-2019-leveraging-rule-based-machine-translation/) of the 140673 papers whose bibliographies we read[Proceedings of Machine Translation Summit XVII: Translator, Project and User Tracks](https://aclanthology.org/volumes/W19-67/)[licence](https://creativecommons.org/licenses/by-nd/4.0/)

- [Publisher's page](https://aclanthology.org/W19-6725/)
- [PDF](https://aclanthology.org/W19-6725.pdf)

**Authors (7)**

1. Daniel Torregrosa
2. Nivranshu Pasricha
3. Maraim Masoud
4. Bharathi Raja Chakravarthi
5. Juan Alonso
6. Noe Casas
7. Mihael Arcan

Citations and author metrics from OpenAlex (); authors with the same name are not always told apart.

## What it is about

The paper takes the knowledge inside a rule-based system (Lucy LT, with its lexicons, morphological classes and parse trees) and feeds it to a neural translator trained on small data, for English to Spanish (generic and medical), Basque, Irish and Simplified Chinese. Three kinds of knowledge are tried. First, the lexicon's ambiguity classes: a surface word gets one tag for each combination of possible category and inflection class, or a tag from an external POS tagger. Second, the parse tree of the source sentence, written as nested brackets and with the phrase label of each word as an extra feature. Third, the resulting morphological features as word features beside subword units (BPE). The results are that the morphological features do about as well as subword units, and that no feature model beat the plain subword baseline by a significant margin. The bracketed tree performed worst, since the model could not learn that serialisation from the amount of data available. The neural baselines beat the rule system and Google Translate on Basque and Irish.

## Ideas

- I1. Keep a set of possible tags, not one. Where the parser or the reader is unsure, a meta token carries a sorted set of candidate UD tags, and the encoder chooses by context. Measure this against a single tag on the Bible dev set, counting the cases where the chosen tag was in the set.
- I2. Do not serialise the full tree for the encoder. The bracket experiment failed on data size, so keep the tree in Rust structures and pass only the head and relation features that the encoder needs.
- I3. Use the phrase type of a token (noun phrase, verb phrase and so on) as a feature of the meta token, as the grandparent-label feature does here. This answers part of open problem 4, since function words and boundaries are then read from the phrase they belong to.
- I4. Test subword fallback as a control. For unseen concepts, the bare form from morph is compared with a subword split of the English word; the paper found the two equivalent for morphology, so the same comparison in Mova decides whether the extra morph code is worth keeping.

## For Mova

The code is tools/translate (the reader's tags in hebrew.rs and the bare-form fallback in profile.rs) and the parser output of tools/macula. The ambiguity-set idea and the phrase-type feature are the most useful parts. The neural models are out of scope; the paper's finding that morphological features and subwords are about equal is a reason to test the morph fallback before extending it.

Topics: [Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/topics/machine-translation/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work.

```
@inproceedings{torregrosa2019leveraging,
  title = {{Leveraging Rule-Based Machine Translation Knowledge for Under-Resourced Neural Machine Translation Models}},
  author = {Torregrosa, Daniel and Pasricha, Nivranshu and Masoud, Maraim and Chakravarthi, Bharathi Raja and Alonso, Juan and Casas, Noe and Arcan, Mihael},
  booktitle = {Proceedings of Machine Translation Summit XVII: Translator, Project and User Tracks},
  year = {2019},
  url = {https://aclanthology.org/W19-6725/}
}
```

- [← previous SYSTRAN](https://md.mova.elamur.ai/free/en/docs/papers/toma-1976-systran/)
- [next → Generation from Lexical Conceptual Structures](https://md.mova.elamur.ai/free/en/docs/papers/traum-2000-generation-lexical-conceptual-structures/)
- [↑ back to Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/mtrans/)

## References

34 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Agerri, Rodrigo, Josu Bermudez, and German Rigau. 2014. IXA pipeline: Efficient and Ready to Use Multilingual NLP tools. In Calzolari, Nicoletta, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, and Stelios Piperidis, editors, Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), Reykjavik, Iceland, may. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Agerri%2C+Rodrigo%2C+Josu+Bermudez%2C+and+German+Rigau%2E+2014%2E+IXA+pipeline%3A+Efficient+and+Ready+to+Use+Multilingual+NLP+tools%2E+In+Calzolari%2C+Nicoletta%2C+Khalid+Choukri%2C+Thierry+Declerck%2C+Hrafn+Loftsson%2C+Bent)
2. Aharoni, Roee and Yoav Goldberg. 2017. Towards String-To-Tree Neural Machine Translation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, volume 2: Short Papers, pages 132–140, Vancouver, Canada, July. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Aharoni%2C+Roee+and+Yoav+Goldberg%2E+2017%2E+Towards+String%2DTo%2DTree+Neural+Machine+Translation%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+volume+2%3A+Short+Pa)
3. Alonso, Juan A and Gregor Thurmair. 2003. The Com-prendium Translator system. In Proceedings of the Ninth Machine Translation Summit. [find](https://scholar.google.com/scholar?q=Alonso%2C+Juan+A+and+Gregor+Thurmair%2E+2003%2E+The+Com%2Dprendium+Translator+system%2E+In+Proceedings+of+the+Ninth+Machine+Translation+Summit%2E)
4. Arcan, Mihael, Caoilfhionn Lane, Eoin Ó Droighneáin, and Paul Buitelaar. 2016. Iris: English-irish machine translation system. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC 2016), Portorož, Slovenia. [find](https://scholar.google.com/scholar?q=Arcan%2C+Mihael%2C+Caoilfhionn+Lane%2C+Eoin+%C3%93+Droighne%C3%A1in%2C+and+Paul+Buitelaar%2E+2016%2E+Iris%3A+English%2Dirish+machine+translation+system%2E+In+Proceedings+of+the+Ninth+International+Conference+on+Language+Resource)
5. Ataman, Duygu and Marcello Federico. 2018. Compositional Representation of Morphologically-Rich Input for Neural Machine Translation. Proceedings of the 56 th Annual Meeting of the Association for Computational Linguistics, pages 305–311. [find](https://scholar.google.com/scholar?q=Ataman%2C+Duygu+and+Marcello+Federico%2E+2018%2E+Compositional+Representation+of+Morphologically%2DRich+Input+for+Neural+Machine+Translation%2E+Proceedings+of+the+56+th+Annual+Meeting+of+the+Association+for+Com)
6. Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. Proceedings of the Third International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Bahdanau%2C+Dzmitry%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2015%2E+Neural+Machine+Translation+by+Jointly+Learning+to+Align+and+Translate%2E+Proceedings+of+the+Third+International+Conference+on+Learning+Represen)
7. Banerjee, Tamali and Pushpak Bhattacharyya. 2018. Meaningless yet meaningful: Morphology grounded subword-level NMT. In Proceedings of the Second Workshop on Subword/Character LEvel Models, pages 55–60, New Orleans, June. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Banerjee%2C+Tamali+and+Pushpak+Bhattacharyya%2E+2018%2E+Meaningless+yet+meaningful%3A+Morphology+grounded+subword%2Dlevel+NMT%2E+In+Proceedings+of+the+Second+Workshop+on+Subword%2FCharacter+LEvel+Models%2C+pages+55%E2%80%936)
8. Baniata, Laith H., Seyoung Park, and Seong-Bae Park. 2018. A Multitask-Based Neural Machine Translation Model with Part-of-Speech Tags Integration for Arabic Dialects. Applied Sciences, 8(12). [find](https://scholar.google.com/scholar?q=Baniata%2C+Laith+H%2E%2C+Seyoung+Park%2C+and+Seong%2DBae+Park%2E+2018%2E+A+Multitask%2DBased+Neural+Machine+Translation+Model+with+Part%2Dof%2DSpeech+Tags+Integration+for+Arabic+Dialects%2E+Applied+Sciences%2C+8%2812%29%2E)
9. Bastings, Joost, Ivan Titov, Wilker Aziz, Diego Marcheggiani, and Khalil Simaan. 2017. Graph Convolutional encoders for syntax-aware neural machine translation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1957–1967, Copenhagen, Denmark, September. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Bastings%2C+Joost%2C+Ivan+Titov%2C+Wilker+Aziz%2C+Diego+Marcheggiani%2C+and+Khalil+Simaan%2E+2017%2E+Graph+Convolutional+encoders+for+syntax%2Daware+neural+machine+translation%2E+In+Proceedings+of+the+2017+Conference+o)
10. Civit, Montserrat and Ma Antònia Martí. 2004. Building cast3lb: A spanish treebank. Research on Language and Computation, 2(4):549–574. [find](https://scholar.google.com/scholar?q=Civit%2C+Montserrat+and+Ma+Ant%C3%B2nia+Mart%C3%AD%2E+2004%2E+Building+cast3lb%3A+A+spanish+treebank%2E+Research+on+Language+and+Computation%2C+2%284%29%3A549%E2%80%93574%2E)
11. Eisele, Andreas and Yu Chen. 2010. MultiUN: A Multilingual Corpus from United Nation Documents. In Tapias, Daniel, Mike Rosner, Stelios Piperidis, Jan Odjik, Joseph Mariani, Bente Maegaard, Khalid Choukri, and Nicoletta Calzolari (Conference Chair), editors, Proceedings of the Seventh conference on International Language Resources and Evaluation, pages 2868–2872. European Language Resources Association (ELRA), 5. [find](https://scholar.google.com/scholar?q=Eisele%2C+Andreas+and+Yu+Chen%2E+2010%2E+MultiUN%3A+A+Multilingual+Corpus+from+United+Nation+Documents%2E+In+Tapias%2C+Daniel%2C+Mike+Rosner%2C+Stelios+Piperidis%2C+Jan+Odjik%2C+Joseph+Mariani%2C+Bente+Maegaard%2C+Khalid+Cho)
12. Eriguchi, Akiko, Kazuma Hashimoto, and Yoshimasa Tsuruoka. 2016. Tree-to-Sequence Attentional Neural Machine Translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, volume 1: Long Papers, pages 823–833. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Eriguchi%2C+Akiko%2C+Kazuma+Hashimoto%2C+and+Yoshimasa+Tsuruoka%2E+2016%2E+Tree%2Dto%2DSequence+Attentional+Neural+Machine+Translation%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computational)
13. Etchegoyhen, Thierry, Eva Martínez Garcia, Andoni Azpeitia, Gorka Labaka, Iñaki Alegria, Itziar Cortes Etxabe, Amaia Jauregi Carrera, Igor Ellakuria Santos, Maite Martin, and Eusebi Calonge. 2018. Neural Machine Translation of Basque. [find](https://scholar.google.com/scholar?q=Etchegoyhen%2C+Thierry%2C+Eva+Mart%C3%ADnez+Garcia%2C+Andoni+Azpeitia%2C+Gorka+Labaka%2C+I%C3%B1aki+Alegria%2C+Itziar+Cortes+Etxabe%2C+Amaia+Jauregi+Carrera%2C+Igor+Ellakuria+Santos%2C+Maite+Martin%2C+and+Eusebi+Calonge%2E+2018%2E+Neu)
14. Forcada, Mikel L, Mireia Ginestí-Rosell, Jacob Nordfalk, Jim O’Regan, Sergio Ortiz-Rojas, Juan Antonio Pérez-Ortiz, Felipe Sánchez-Martínez, Gema Ramírez-Sánchez, and Francis M Tyers. 2011. Apertium: a free/open-source platform for rule-based machine translation. Machine translation, 25(2):127– 144. [find](https://scholar.google.com/scholar?q=Forcada%2C+Mikel+L%2C+Mireia+Ginest%C3%AD%2DRosell%2C+Jacob+Nordfalk%2C+Jim+O%E2%80%99Regan%2C+Sergio+Ortiz%2DRojas%2C+Juan+Antonio+P%C3%A9rez%2DOrtiz%2C+Felipe+S%C3%A1nchez%2DMart%C3%ADnez%2C+Gema+Ram%C3%ADrez%2DS%C3%A1nchez%2C+and+Francis+M+Tyers%2E+2011%2E+Apertium%3A+)
15. Garcıa-Martınez, Mercedes, Loıc Barrault, and Fethi Bougares. 2016. Factored Neural Machine Translation Architectures. In International Workshop on Spoken Language Translation (IWSLT’16), Seattle, US. [find](https://scholar.google.com/scholar?q=Garc%C4%B1a%2DMart%C4%B1nez%2C+Mercedes%2C+Lo%C4%B1c+Barrault%2C+and+Fethi+Bougares%2E+2016%2E+Factored+Neural+Machine+Translation+Architectures%2E+In+International+Workshop+on+Spoken+Language+Translation+%28IWSLT%E2%80%9916%29%2C+Seattle%2C+US%2E)
16. Klein, Guillaume, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M. Rush. 2017. Open-NMT: Open-Source Toolkit for Neural Machine Translation. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, System Demonstrations:67–72. [find](https://scholar.google.com/scholar?q=Klein%2C+Guillaume%2C+Yoon+Kim%2C+Yuntian+Deng%2C+Jean+Senellart%2C+and+Alexander+M%2E+Rush%2E+2017%2E+Open%2DNMT%3A+Open%2DSource+Toolkit+for+Neural+Machine+Translation%2E+Proceedings+of+the+55th+Annual+Meeting+of+the+Assoc)
17. Koehn, Philipp, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, volume 1, pages 48–54. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Koehn%2C+Philipp%2C+Franz+Josef+Och%2C+and+Daniel+Marcu%2E+2003%2E+Statistical+phrase%2Dbased+translation%2E+In+Proceedings+of+the+2003+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+)
18. Koehn, Philipp. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of the 2004 conference on empirical methods in natural language processing. [find](https://scholar.google.com/scholar?q=Koehn%2C+Philipp%2E+2004%2E+Statistical+significance+tests+for+machine+translation+evaluation%2E+In+Proceedings+of+the+2004+conference+on+empirical+methods+in+natural+language+processing%2E)
19. Koehn, Philipp. 2005. Europarl: A Parallel Corpus for Statistical Machine Translation. In Conference Proceedings: the tenth Machine Translation Summit. AAMT. [find](https://scholar.google.com/scholar?q=Koehn%2C+Philipp%2E+2005%2E+Europarl%3A+A+Parallel+Corpus+for+Statistical+Machine+Translation%2E+In+Conference+Proceedings%3A+the+tenth+Machine+Translation+Summit%2E+AAMT%2E)
20. Lison, Pierre and Jörg Tiedemann. 2016. OpenSubtitles2016: Extracting Large Parallel Corpora from Movie and TV Subtitles. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Paris, France, may. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Lison%2C+Pierre+and+J%C3%B6rg+Tiedemann%2E+2016%2E+OpenSubtitles2016%3A+Extracting+Large+Parallel+Corpora+from+Movie+and+TV+Subtitles%2E+In+Proceedings+of+the+Tenth+International+Conference+on+Language+Resources+and)
21. Marcus, Mitchell, Grace Kim, Mary Ann Marcinkiewicz, Robert MacIntyre, Ann Bies, Mark Ferguson, Karen Katz, and Britta Schasberger. 1994. The Penn Treebank: Annotating Predicate Argument Structure. In Proceedings of the Workshop on Human Language Technology, HLT ’94, pages 114–119, Stroudsburg, PA, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Marcus%2C+Mitchell%2C+Grace+Kim%2C+Mary+Ann+Marcinkiewicz%2C+Robert+MacIntyre%2C+Ann+Bies%2C+Mark+Ferguson%2C+Karen+Katz%2C+and+Britta+Schasberger%2E+1994%2E+The+Penn+Treebank%3A+Annotating+Predicate+Argument+Structure%2E+In) [our card](https://md.mova.elamur.ai/free/en/docs/papers/bank-1994-marcus-penn-treebank-annotating-predicate-argument/)
22. Nadejde, Maria, Siva Reddy, Rico Sennrich, Tomasz Dwojak, Marcin Junczys-Dowmunt, Philipp Koehn, and Alexandra Birch. 2017. Predicting target language ccg supertags improves neural machine translation. In Proceedings of the Second Conference on Machine Translation, pages 68–79, Copenhagen, Denmark, September. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Nadejde%2C+Maria%2C+Siva+Reddy%2C+Rico+Sennrich%2C+Tomasz+Dwojak%2C+Marcin+Junczys%2DDowmunt%2C+Philipp+Koehn%2C+and+Alexandra+Birch%2E+2017%2E+Predicting+target+language+ccg+supertags+improves+neural+machine+translation)
23. Niehues, Jan and Eunah Cho. 2017. Exploiting Linguistic Resources for Neural Machine Translation Using Multi-task Learning. In Proceedings of the Second Conference on Machine Translation, pages 80–89, Copenhagen, Denmark, September. Association for Computational Linguistics. Nivre, Joakim, Mitchell Abrams, Željko Agić, Lars Ahrenberg, Lene Antonsen, Katya Aplonova, Maria Jesus Aranzabe, Gashaw Arutie, Masayuki Asahara, Luma Ateyah, Mohammed Attia, Aitziber Atutxa, Liesbeth Augustinus, Elena Badmaeva, Miguel Ballesteros, Esha Banerjee, Sebastian Bank, Verginica Barbu Mititelu, Victoria Basmov, John Bauer, Sandra Bellato, Kepa Bengoetxea, Yevgeni Berzak, Irshad Ahmad Bhat, Riyaz Ahmad Bhat, Erica Biagetti, Eckhard Bick, Rogier Blokland, Victoria Bobicev, Carl Börstell, Cristina Bosco, Gosse Bouma, Sam Bowman, Adriane Boyd, Aljoscha Burchardt, Marie Candito, Bernard Caron, Gauthier Caron, Gülşen Cebiroğlu Eryiğit, Flavio Massimiliano Cecchini, Giuseppe G. A. Celano, Slavomír Čéplö, Savas Cetin, Fabricio Chalub, Jinho Choi, Yongseok Cho, Jayeol Chun, Silvie Cinková, Aurélie Collomb, Çağrı Çöltekin, Miriam Connor, Marine Courtin, Elizabeth Davidson, Marie-Catherine de Marneffe, Valeria de Paiva, Arantza Diaz de Ilarraza, Carly Dickerson, Peter Dirix, Kaja Dobrovoljc, Timothy Dozat, Kira Droganova, Puneet Dwivedi, Marhaba Eli, Ali Elkahky, Binyam Ephrem, Tomaž Erjavec, Aline Etienne, Richárd Farkas, Hector Fernandez Alcalde, Jennifer Foster, Cláudia Freitas, Katarína Gajdošová, Daniel Galbraith, Marcos Garcia, Moa Gärdenfors, Sebastian Garza, Kim Gerdes, Filip Ginter, Iakes Goenaga, Koldo Gojenola, Memduh Gökırmak, Yoav Goldberg, Xavier Gómez Guinovart, Berta Gonzáles Saavedra, Matias Grioni, Normunds Grūzı̄tis, Bruno Guillaume, Céline Guillot-Barbance, Nizar Habash, Jan Hajič, Jan Hajič jr., Linh Hà Mỹ, Na-Rae Han, Kim Harris, Dag Haug, Barbora Hladká, Jaroslava Hlaváčová, Florinel Hociung, Petter Hohle, Jena Hwang, Radu Ion, Elena Irimia, O.lájídé Ishola, Tomáš Jelínek, Anders Johannsen, Fredrik Jørgensen, Hüner Kaşıkara, Sylvain Kahane, Hiroshi Kanayama, Jenna Kanerva, Boris Katz, Tolga Kayadelen, Jessica Kenney, Václava Kettnerová, Jesse Kirchner, Kamil Kopacewicz, Natalia Kotsyba, Simon Krek, Sookyoung Kwak, Veronika Laippala, Lorenzo Lambertino, Lucia Lam, Tatiana Lando, Septina Dian Larasati, Alexei Lavrentiev, John Lee, Phương Lê Hồng, Alessandro Lenci, Saran Lertpradit, Herman Leung, Cheuk Ying Li, Josie Li, Keying Li, KyungTae Lim, Nikola Ljubešić, Olga Loginova, Olga Lyashevskaya, Teresa Lynn, Vivien Macketanz, Aibek Makazhanov, Michael Mandl, Christopher Manning, Ruli Manurung, Cătălina Mărănduc, David Mareček, Katrin Marheinecke, Héctor Martínez Alonso, André Martins, Jan Mašek, Yuji Matsumoto, Ryan McDonald, Gustavo Mendonça, Niko Miekka, Margarita Misirpashayeva, Anna Missilä, Cătălin Mititelu, Yusuke Miyao, Simonetta Montemagni, Amir More, Laura Moreno Romero, Keiko Sophie Mori, Shinsuke Mori, Bjartur Mortensen, Bohdan Moskalevskyi, Kadri Muischnek, Yugo Murawaki, Kaili Müürisep, Pinkey Nainwani, Juan Ignacio Navarro Horñiacek, Anna Nedoluzhko, Gunta Nešpore-Bērzkalne, Lương Nguyễn Thi., Huyền Nguyễn Thi. Minh, Vitaly Nikolaev, Rattima Nitisaroj, Hanna Nurmi, Stina Ojala, Adédayò. Olúòkun, Mai Omura, Petya Osenova, Robert Östling, Lilja Øvrelid, Niko Partanen, Elena Pascual, Marco Passarotti, Agnieszka Patejuk, Guilherme Paulino-Passos, Siyao Peng, Cenel-Augusto Perez, Guy Perrier, Slav Petrov, Jussi Piitulainen, Emily Pitler, Barbara Plank, Thierry Poibeau, Martin Popel, Lauma Pretkalnin , a, Sophie Prévost, Prokopis Prokopidis, Adam Przepiórkowski, Tiina Puolakainen, Sampo Pyysalo, Andriela Rääbis, Alexandre Rademaker, Loganathan Ramasamy, Taraka Rama, Carlos Ramisch, Vinit Ravishankar, Livy Real, Siva Reddy, Georg Rehm, Michael Rießler, Larissa Rinaldi, Laura Rituma, Luisa Rocha, Mykhailo Romanenko, Rudolf Rosa, Davide Rovati, Valentin Ros , ca, Olga Rudina, Jack Rueter, Shoval Sadde, Benoît Sagot, Shadi Saleh, Tanja Samardžić, Stephanie Samson, Manuela Sanguinetti, Baiba Saulı̄te, Yanin Sawanakunanon, Nathan Schneider, Sebastian Schuster, Djamé Seddah, Wolfgang Seeker, Mojgan Seraji, Mo Shen, Atsuko Shimada, Muh Shohibussirri, Dmitry Sichinava, Natalia Silveira, Maria Simi, Radu Simionescu, Katalin Simkó, Mária Šimková, Kiril Simov, Aaron Smith, Isabela Soares-Bastos, Carolyn Spadine, Antonio Stella, Milan Straka, Jana Strnadová, Alane Suhr, Umut Sulubacak, Zsolt Szántó, Dima Taji, Yuta Takahashi, Takaaki Tanaka, Isabelle Tellier, Trond Trosterud, Anna Trukhina, Reut Tsarfaty, Francis Tyers, Sumire Uematsu, Zdeňka Urešová, Larraitz Uria, Hans Uszkoreit, Sowmya Vajjala, Daniel van Niekerk, Gertjan van Noord, Viktor Varga, Eric Villemonte de la Clergerie, Veronika Vincze, Lars Wallin, Jing Xian Wang, Jonathan North Washington, Seyi Williams, Mats Wirén, Tsegay Woldemariam, Tak-sum Wong, Chunxiao Yan, Marat M. Yavrumyan, Zhuoran Yu, Zdeněk Žabokrtský, Amir Zeldes, Daniel Zeman, Manying Zhang, and Hanzhi Zhu. 2018. Universal Dependencies 2.3. LIN-DAT/CLARIN digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University. [find](https://scholar.google.com/scholar?q=Niehues%2C+Jan+and+Eunah+Cho%2E+2017%2E+Exploiting+Linguistic+Resources+for+Neural+Machine+Translation+Using+Multi%2Dtask+Learning%2E+In+Proceedings+of+the+Second+Conference+on+Machine+Translation%2C+pages+80%E2%80%9389%2C)
24. Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Papineni%2C+Kishore%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+BLEU%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+annual+meeting+on+association+for+computa) [our card](https://md.mova.elamur.ai/free/en/docs/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
25. Sennrich, Rico and Barry Haddow. 2016. Linguistic Input Features Improve Neural Machine Translation. In Proceedings of the First Conference on Machine Translation, volume 1: Research Papers, pages 83– 91. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sennrich%2C+Rico+and+Barry+Haddow%2E+2016%2E+Linguistic+Input+Features+Improve+Neural+Machine+Translation%2E+In+Proceedings+of+the+First+Conference+on+Machine+Translation%2C+volume+1%3A+Research+Papers%2C+pages+83%E2%80%93)
26. Sennrich, Rico, Barry Haddow, and Alexandra Birch. 2016. Neural Machine Translation of Rare Words with Subword Units. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, abs/1508.07909. [↗](https://arxiv.org/abs/1508.07909)
27. Shi, Xing, Inkit Padhi, and Kevin Knight. 2016. Does String-Based Neural MT Learn Source Syntax? In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1526–1534, Austin, Texas, November. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Shi%2C+Xing%2C+Inkit+Padhi%2C+and+Kevin+Knight%2E+2016%2E+Does+String%2DBased+Neural+MT+Learn+Source+Syntax%3F+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+1526%E2%80%931)
28. Snover, Matthew, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of association for machine translation in the Americas, volume 200. [find](https://scholar.google.com/scholar?q=Snover%2C+Matthew%2C+Bonnie+Dorr%2C+Richard+Schwartz%2C+Linnea+Micciulla%2C+and+John+Makhoul%2E+2006%2E+A+study+of+translation+edit+rate+with+targeted+human+annotation%2E+In+Proceedings+of+association+for+machine+tra) [our card](https://md.mova.elamur.ai/free/en/docs/papers/snover-2006-study-translation-edit-rate-targeted/)
29. Steinberger, Ralf, Mohamed Ebrahim, Alexandros Poulis, Manuel Carrasco-Benitez, Patrick Schlüter, Marek Przybyszewski, and Signe Gilbro. 2014. An overview of the european union’s highly multilingual parallel corpora. Language Resources and Evaluation, 48(4):679–707. [find](https://scholar.google.com/scholar?q=Steinberger%2C+Ralf%2C+Mohamed+Ebrahim%2C+Alexandros+Poulis%2C+Manuel+Carrasco%2DBenitez%2C+Patrick+Schl%C3%BCter%2C+Marek+Przybyszewski%2C+and+Signe+Gilbro%2E+2014%2E+An+overview+of+the+european+union%E2%80%99s+highly+multilingual+p)
30. Tiedemann, Jorg. 2009. News from OPUS – A Collection of Multilingual Parallel Corpora with Tools and Interfaces. In Advances in Natural Language Processing, volume 5, pages 237–248. Borovets, Bulgaria. [find](https://scholar.google.com/scholar?q=Tiedemann%2C+Jorg%2E+2009%2E+News+from+OPUS+%E2%80%93+A+Collection+of+Multilingual+Parallel+Corpora+with+Tools+and+Interfaces%2E+In+Advances+in+Natural+Language+Processing%2C+volume+5%2C+pages+237%E2%80%93248%2E+Borovets%2C+Bulgaria)
31. Tiedemann, Jörg. 2012. Parallel data, tools and interfaces in opus. In Calzolari, Nicoletta, Khalid Choukri, Thierry Declerck, Mehmet Uğur Doğan, Bente Maegaard, Joseph Mariani, Jan Odijk, and Stelios Piperidis, editors, Proceedings of the Eight International Conference on Language Resources and Evaluation, Istanbul, Turkey, may. [find](https://scholar.google.com/scholar?q=Tiedemann%2C+J%C3%B6rg%2E+2012%2E+Parallel+data%2C+tools+and+interfaces+in+opus%2E+In+Calzolari%2C+Nicoletta%2C+Khalid+Choukri%2C+Thierry+Declerck%2C+Mehmet+Ug%CC%86ur+Dog%CC%86an%2C+Bente+Maegaard%2C+Joseph+Mariani%2C+Jan+Odijk%2C+and+Steli)
32. Toma, Peter. 1977. Systran as a multilingual machine translation system. In Proceedings of the Third European Congress on Information Systems and Networks, Overcoming the language barrier, pages 569– 581. [find](https://scholar.google.com/scholar?q=Toma%2C+Peter%2E+1977%2E+Systran+as+a+multilingual+machine+translation+system%2E+In+Proceedings+of+the+Third+European+Congress+on+Information+Systems+and+Networks%2C+Overcoming+the+language+barrier%2C+pages+569%E2%80%93+)
33. Toutanova, Kristina, Dan Klein, Christopher D Manning, and Yoram Singer. 2003. Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, volume 1, pages 173–180. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Toutanova%2C+Kristina%2C+Dan+Klein%2C+Christopher+D+Manning%2C+and+Yoram+Singer%2E+2003%2E+Feature%2Drich+part%2Dof%2Dspeech+tagging+with+a+cyclic+dependency+network%2E+In+Proceedings+of+the+2003+Conference+of+the+North+)
34. Zoph, Barret and Kevin Knight. 2016. Multi-Source Neural Translation. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 30–34, San Diego, California, June. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zoph%2C+Barret+and+Kevin+Knight%2E+2016%2E+Multi%2DSource+Neural+Translation%2E+In+Proceedings+of+the+2016+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%3A+Human+Langua)