[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/grunewald-2021-applying-occam-s-razor-transformer/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / Applying Occam's Razor to Transformer-Based Dependency Parsing: What Works, What Doesn't, and What is Really Necessary

# Applying Occam's Razor to Transformer-Based Dependency Parsing: What Works, What Doesn't, and What is Really Necessary

**Authors:** [Stefan Grünewald](https://md.mova.elamur.ai/free/en/docs/papers/authors/stefan-grunewald/), [Annemarie Friedrich](https://md.mova.elamur.ai/free/en/docs/papers/authors/annemarie-friedrich/), [Jonas Kuhn](https://md.mova.elamur.ai/free/en/docs/papers/authors/jonas-kuhn/) · **Year:** 2021 · **Venue:** Proceedings of the 17th International Conference on Parsing Technologies and the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies (IWPT 2021) **Link:** [aclanthology.org/2021.iwpt-1.13](https://aclanthology.org/2021.iwpt-1.13/) (ACL Anthology 2021.iwpt-1.13; DOI [10.18653/v1/2021.iwpt-1.13](https://doi.org/10.18653/v1/2021.iwpt-1.13)) **License of the paper:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) — [creativecommons.org/licenses/by/4.0](https://creativecommons.org/licenses/by/4.0/)

[**9** citations](https://openalex.org/works?filter=cites:W3138336861) openalex-snapshot · 2026-10-14[**9** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/grunewald-2021-applying-occam-s-razor-transformer/) of the 140673 papers whose bibliographies we read[Proceedings of the 17th International Conference on Parsing Technologies and the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies (IWPT 2021)](https://aclanthology.org/volumes/2021.iwpt-1/)[licence](https://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://aclanthology.org/2021.iwpt-1.13/)
- [PDF](https://aclanthology.org/2021.iwpt-1.13.pdf)
- [DOI 10.18653/v1/2021.iwpt-1.13](https://doi.org/10.18653/v1/2021.iwpt-1.13)
- [OpenAlex](https://openalex.org/W3138336861)

**Authors (3)**

1. Stefan Grünewald [Semantic Scholar](https://www.semanticscholar.org/author/49948196)
2. [Annemarie Friedrich](https://openalex.org/A5029262106) [Semantic Scholar](https://www.semanticscholar.org/author/33985877) · h 13 · 700 citations · 83 works
3. [Jonas Kuhn](https://openalex.org/A5101531241) [ORCID](https://orcid.org/0000-0003-2860-5960) · [Semantic Scholar](https://www.semanticscholar.org/author/1716963) · h 22 · 1543 citations · 160 works

Citations and author metrics from openalex-snapshot (2026-10-14); authors with the same name are not always told apart.

## Summary

Modern graph-based dependency parsers for Universal Dependencies rely on transformer embeddings but differ in several design choices: which pretrained model to use, whether to add LSTM layers, and whether to train jointly on POS and morphology. Following Occam's razor, the authors ask which of these components are actually needed. They build STEPS, a modular graph-based parser with biaffine classifiers, and run systematic experiments on UD treebanks of twelve typologically diverse languages. They compare mBERT, language-specific BERTs and XLM-R, with and without a BiLSTM, and with and without multi-task training. The choice of pretrained model matters most, and XLM-R is the most reliable. The LSTM adds nothing, and multi-task training can hurt and can distort comparisons between systems. A simple setup, fine-tuned XLM-R with factorised arc and label scoring, sets a new LAS state of the art for most languages and carries over well to Enhanced UD. The practical lesson is that architectural add-ons are often unnecessary; what matters is a strong language model that covers the target language and domain.

## How Mova uses it

- the development notes of `en`: the STEPS (XLM-R) result on English-EWT test, 91.91 LAS as reported in the paper, is one of the reference points in the benchmark table that Mova's own tagger and parser are measured against.
- Mova does not reuse STEPS's architecture: it has no transformer and no neural parser. The paper serves as an external yardstick for what a strong neural parser reaches on the same test set.

## Effectiveness in Mova

Used only as a benchmark reference, so there is nothing to measure for the paper itself. The comparison as recorded in the development notes of `en` (EWT test, gold tokens): STEPS (XLM-R) 91.91 LAS; UDPipe 2 (EWT 2.17 model, Mova's own measurement) 92.37 LAS; Mova small model plus large-model edits 91.90 LAS on 300 random sentences, where UDPipe scores 92.11; Mova's small model alone 80.24 LAS, and 81.15 LAS when also trained on GUM converted to Mova's annotation dialect.

---

👨‍🔬💥

Topics: [Dependency parsing](https://md.mova.elamur.ai/free/en/docs/papers/topics/dependency-parsing/) · [Universal Dependencies](https://md.mova.elamur.ai/free/en/docs/papers/topics/universal-dependencies/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.18653/v1/2021.iwpt-1.13](https://doi.org/10.18653/v1/2021.iwpt-1.13)

```
@inproceedings{grunewald2021applying,
  title = {{Applying Occam's Razor to Transformer-Based Dependency Parsing: What Works, What Doesn't, and What is Really Necessary}},
  author = {Grünewald, Stefan and Friedrich, Annemarie and Kuhn, Jonas},
  booktitle = {Proceedings of the 17th International Conference on Parsing Technologies and the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies (IWPT 2021)},
  year = {2021},
  doi = {10.18653/v1/2021.iwpt-1.13},
  url = {https://doi.org/10.18653/v1/2021.iwpt-1.13}
}
```

- [← previous LLM Dependency Parsing with In-Context Rules](https://md.mova.elamur.ai/free/en/docs/papers/ginn-2025-llm-dependency-parsing-context-rules/)
- [next → Coordinate Constructions in English Enhanced Universal Dependencies: Analysis and Computational Modeling](https://md.mova.elamur.ai/free/en/docs/papers/grunewald-2021-coordinate-constructions-english-enhanced-universal/)
- [↑ back to papers](https://md.mova.elamur.ai/free/en/docs/papers/)

## References

56 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. pages 8440–8451. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Kartikay+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzm%C3%A1n%2C+Edouard+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2020%2E+Unsupervised+cross%2Dlingu)
2. Mikhail Arkhipov, Maria Trofimova, Yuri Kuratov, and Alexey Sorokin. 2019. Tuning multilingual transformers for language-specific named entity recognition. In Proceedings of the 7th Workshop on Balto-Slavic Natural Language Processing, pages 89–93, Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Krist… [find](https://scholar.google.com/scholar?q=Mikhail+Arkhipov%2C+Maria+Trofimova%2C+Yuri+Kuratov%2C+and+Alexey+Sorokin%2E+2019%2E+Tuning+multilingual+transformers+for+language%2Dspecific+named+entity+recognition%2E+In+Proceedings+of+the+7th+Workshop+on+Balto%2D)
3. Timothy Dozat and Christopher D. Manning. 2017. Deep biaffine attention for neural dependency parsing. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. Open-Review.net. [find](https://scholar.google.com/scholar?q=Timothy+Dozat+and+Christopher+D%2E+Manning%2E+2017%2E+Deep+biaffine+attention+for+neural+dependency+parsing%2E+In+5th+International+Conference+on+Learning+Representations%2C+ICLR+2017%2C+Toulon%2C+France%2C+April+24%2D) [our card](https://md.mova.elamur.ai/free/en/docs/papers/missing-2016-dozat-deep-biaffine-attention-neural-dependency/)
4. Timothy Dozat and Christopher D. Manning. 2018. Simpler but more accurate semantic dependency parsing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 484–490, Melbourne, Australia. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Timothy+Dozat+and+Christopher+D%2E+Manning%2E+2018%2E+Simpler+but+more+accurate+semantic+dependency+parsing%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volum)
5. Timothy Dozat, Peng Qi, and Christopher D. Manning. 2017. Stanford’s graph-based neural dependency parser at the CoNLL 2017 shared task. In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 20–30, Vancouver, Canada. Association for Computa… [find](https://scholar.google.com/scholar?q=Timothy+Dozat%2C+Peng+Qi%2C+and+Christopher+D%2E+Manning%2E+2017%2E+Stanford%E2%80%99s+graph%2Dbased+neural+dependency+parser+at+the+CoNLL+2017+shared+task%2E+In+Proceedings+of+the+CoNLL+2017+Shared+Task%3A+Multilingual+Pars) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2017-dozat-stanford-s-graph-based-neural/)
6. Kira Droganova, Olga Lyashevskaya, and Daniel Zeman. 2018. Data conversion and consistency of monolingual corpora: Russian ud treebanks. In Proceedings of the 17th International Workshop on Treebanks and Linguistic Theories (TLT 2018), December 13–14, 2018, Oslo University, Norway, 155, pages 52–65.… [find](https://scholar.google.com/scholar?q=Kira+Droganova%2C+Olga+Lyashevskaya%2C+and+Daniel+Zeman%2E+2018%2E+Data+conversion+and+consistency+of+monolingual+corpora%3A+Russian+ud+treebanks%2E+In+Proceedings+of+the+17th+International+Workshop+on+Treebanks+)
7. Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A. Smith. 2015. Transition-based dependency parsing with stack long short-term memory. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural… [find](https://scholar.google.com/scholar?q=Chris+Dyer%2C+Miguel+Ballesteros%2C+Wang+Ling%2C+Austin+Matthews%2C+and+Noah+A%2E+Smith%2E+2015%2E+Transition%2Dbased+dependency+parsing+with+stack+long+short%2Dterm+memory%2E+In+Proceedings+of+the+53rd+Annual+Meeting+of) [our card](https://md.mova.elamur.ai/free/en/docs/papers/missing-2015-dyer-transition-based-dependency-parsing-stack/)
8. Jack Edmonds. 1967. Optimum branchings. Journal of Research of the national Bureau of Standards B, 71(4):233–240. [find](https://scholar.google.com/scholar?q=Jack+Edmonds%2E+1967%2E+Optimum+branchings%2E+Journal+of+Research+of+the+national+Bureau+of+Standards+B%2C+71%284%29%3A233%E2%80%93240%2E)
9. Agnieszka Falenska and Jonas Kuhn. 2019. The (non-)utility of structural features in BiLSTM-based dependency parsers. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 117–128, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Agnieszka+Falenska+and+Jonas+Kuhn%2E+2019%2E+The+%28non%2D%29utility+of+structural+features+in+BiLSTM%2Dbased+dependency+parsers%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Lin)
10. Erick Fonseca and André F. T. Martins. 2020. Revisiting higher-order dependency parsers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8795– 8800, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Erick+Fonseca+and+Andr%C3%A9+F%2E+T%2E+Martins%2E+2020%2E+Revisiting+higher%2Dorder+dependency+parsers%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+8795%E2%80%93+8800%2C+O)
11. Johannes Gontrum, Jonas Groschwitz, Alexander Koller, and Christoph Teichmann. 2017. Alto: Rapid prototyping for parsing and translation. In Proceedings of the Software Demonstrations of the 15th Conference of the European Chapter of the Association for Computational Linguistics, pages 29– 32, Valen… [find](https://scholar.google.com/scholar?q=Johannes+Gontrum%2C+Jonas+Groschwitz%2C+Alexander+Koller%2C+and+Christoph+Teichmann%2E+2017%2E+Alto%3A+Rapid+prototyping+for+parsing+and+translation%2E+In+Proceedings+of+the+Software+Demonstrations+of+the+15th+Conf) [our card](https://md.mova.elamur.ai/free/en/docs/papers/gontrum-2017-alto-rapid-prototyping-parsing-translation/)
12. Stefan Grünewald and Annemarie Friedrich. 2020. RobertNLP at the IWPT 2020 shared task: Surprisingly simple enhanced UD parsing for English. In Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies, page… [find](https://scholar.google.com/scholar?q=Stefan+Gr%C3%BCnewald+and+Annemarie+Friedrich%2E+2020%2E+RobertNLP+at+the+IWPT+2020+shared+task%3A+Surprisingly+simple+enhanced+UD+parsing+for+English%2E+In+Proceedings+of+the+16th+International+Conference+on+Pars) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2020-grunewald-robertnlp-iwpt-2020-shared-task/)
13. Katri Haverinen, Jenna Nyblom, Timo Viljanen, Veronika Laippala, Samuel Kohonen, Anna Missilä, Stina Ojala, Tapio Salakoski, and Filip Ginter. 2014. Building the essential resources for finnish: the turku dependency treebank. Language Resources and Evaluation, 48(3):493–531. [find](https://scholar.google.com/scholar?q=Katri+Haverinen%2C+Jenna+Nyblom%2C+Timo+Viljanen%2C+Veronika+Laippala%2C+Samuel+Kohonen%2C+Anna+Missil%C3%A4%2C+Stina+Ojala%2C+Tapio+Salakoski%2C+and+Filip+Ginter%2E+2014%2E+Building+the+essential+resources+for+finnish%3A+the+t)
14. John Hewitt and Christopher D. Manning. 2019. A structural probe for finding syntax in word representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4129… [find](https://scholar.google.com/scholar?q=John+Hewitt+and+Christopher+D%2E+Manning%2E+2019%2E+A+structural+probe+for+finding+syntax+in+word+representations%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter+of+the+Association+for+)
15. Jenna Kanerva, Filip Ginter, Niko Miekka, Akseli Leino, and Tapio Salakoski. 2018. Turku neural parser pipeline: An end-to-end system for the CoNLL 2018 shared task. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 133–142, Brussels, B… [find](https://scholar.google.com/scholar?q=Jenna+Kanerva%2C+Filip+Ginter%2C+Niko+Miekka%2C+Akseli+Leino%2C+and+Tapio+Salakoski%2E+2018%2E+Turku+neural+parser+pipeline%3A+An+end%2Dto%2Dend+system+for+the+CoNLL+2018+shared+task%2E+In+Proceedings+of+the+CoNLL+2018+S)
16. Jenna Kanerva, Filip Ginter, and Sampo Pyysalo. 2020. Turku enhanced parser pipeline: From raw text to enhanced graphs in the IWPT 2020 shared task. In Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependenci… [find](https://scholar.google.com/scholar?q=Jenna+Kanerva%2C+Filip+Ginter%2C+and+Sampo+Pyysalo%2E+2020%2E+Turku+enhanced+parser+pipeline%3A+From+raw+text+to+enhanced+graphs+in+the+IWPT+2020+shared+task%2E+In+Proceedings+of+the+16th+International+Conference)
17. Eliyahu Kiperwasser and Yoav Goldberg. 2016. Simple and accurate dependency parsing using bidirectional lstm feature representations. Transactions of the Association for Computational Linguistics, 4:313–327. [find](https://scholar.google.com/scholar?q=Eliyahu+Kiperwasser+and+Yoav+Goldberg%2E+2016%2E+Simple+and+accurate+dependency+parsing+using+bidirectional+lstm+feature+representations%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+4%3A3) [our card](https://md.mova.elamur.ai/free/en/docs/papers/narrative-2016-kiperwasser-simple-accurate-dependency-parsing-bidirectional/)
18. Dan Kondratyuk and Milan Straka. 2019. 75 languages, 1 model: Parsing universal dependencies universally. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2779–279… [find](https://scholar.google.com/scholar?q=Dan+Kondratyuk+and+Milan+Straka%2E+2019%2E+75+languages%2C+1+model%3A+Parsing+universal+dependencies+universally%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2019-kondratyuk-75-languages-1-model-parsing/)
19. Artur Kulmizev, Miryam de Lhoneux, Johannes Gontrum, Elena Fano, and Joakim Nivre. 2019. Deep contextualized word embeddings in transition-based and graph-based dependency parsing - a tale of two parsers revisited. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Proces… [find](https://scholar.google.com/scholar?q=Artur+Kulmizev%2C+Miryam+de+Lhoneux%2C+Johannes+Gontrum%2C+Elena+Fano%2C+and+Joakim+Nivre%2E+2019%2E+Deep+contextualized+word+embeddings+in+transition%2Dbased+and+graph%2Dbased+dependency+parsing+%2D+a+tale+of+two+pars)
20. Yuri Kuratov and Mikhail Arkhipov. 2019. Adaptation of deep bidirectional multilingual transformers for russian language. arXiv preprint arXiv:1905.07213. [find](https://scholar.google.com/scholar?q=Yuri+Kuratov+and+Mikhail+Arkhipov%2E+2019%2E+Adaptation+of+deep+bidirectional+multilingual+transformers+for+russian+language%2E+arXiv+preprint+arXiv%3A1905%2E07213%2E)
21. Robin Kurtz, Stephan Oepen, and Marco Kuhlmann. 2020. End-to-end negation resolution as graph parsing. In Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies, pages 14–24, Online. Association for Compu… [find](https://scholar.google.com/scholar?q=Robin+Kurtz%2C+Stephan+Oepen%2C+and+Marco+Kuhlmann%2E+2020%2E+End%2Dto%2Dend+negation+resolution+as+graph+parsing%2E+In+Proceedings+of+the+16th+International+Conference+on+Parsing+Technologies+and+the+IWPT+2020+Sha)
22. Sangah Lee, Hansol Jang, Yunmee Baik, Suzi Park, and Hyopil Shin. 2020. Kr-bert: A small-scale korean-specific language model. arXiv preprint arXiv:2008.03979. — [arXiv:2008.03979](https://arxiv.org/abs/2008.03979) [find](https://scholar.google.com/scholar?q=Sangah+Lee%2C+Hansol+Jang%2C+Yunmee+Baik%2C+Suzi+Park%2C+and+Hyopil+Shin%2E+2020%2E+Kr%2Dbert%3A+A+small%2Dscale+korean%2Dspecific+language+model%2E+arXiv+preprint+arXiv%3A2008%2E03979%2E+%E2%80%94+%5BarXiv%3A2008%2E03979%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
23. Miryam de Lhoneux, Sara Stymne, and Joakim Nivre. 2017. Old school vs. new school: Comparing transition-based parsers with and without neural network enhancement. In The 15th Treebanks and Linguistic Theories Workshop (TLT), pages 99–110. [find](https://scholar.google.com/scholar?q=Miryam+de+Lhoneux%2C+Sara+Stymne%2C+and+Joakim+Nivre%2E+2017%2E+Old+school+vs%2E+new+school%3A+Comparing+transition%2Dbased+parsers+with+and+without+neural+network+enhancement%2E+In+The+15th+Treebanks+and+Linguistic+)
24. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2E+a)
25. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+2019%2E+Decoupled+weight+decay+regularization%2E+In+International+Conference+on+Learning+Representations%2E)
26. Marie-Catherine de Marneffe, Timothy Dozat, Natalia Silveira, Katri Haverinen, Filip Ginter, Joakim Nivre, and Christopher D. Manning. 2014. Universal Stanford dependencies: A cross-linguistic typology. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’1… [find](https://scholar.google.com/scholar?q=Marie%2DCatherine+de+Marneffe%2C+Timothy+Dozat%2C+Natalia+Silveira%2C+Katri+Haverinen%2C+Filip+Ginter%2C+Joakim+Nivre%2C+and+Christopher+D%2E+Manning%2E+2014%2E+Universal+Stanford+dependencies%3A+A+cross%2Dlinguistic+typolog) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2014-marneffe-universal-stanford-dependencies-cross-linguistic/)
27. Ryan McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith Hall, Slav Petrov, Hao Zhang, Oscar Täckström, Claudia Bedini, Núria Bertomeu Castelló, and Jungmee Lee. 2013. Universal Dependency annotation for multilingual parsing. In Proceedings of the 51… [find](https://scholar.google.com/scholar?q=Ryan+McDonald%2C+Joakim+Nivre%2C+Yvonne+Quirmbach%2DBrundage%2C+Yoav+Goldberg%2C+Dipanjan+Das%2C+Kuzman+Ganchev%2C+Keith+Hall%2C+Slav+Petrov%2C+Hao+Zhang%2C+Oscar+T%C3%A4ckstr%C3%B6m%2C+Claudia+Bedini%2C+N%C3%BAria+Bertomeu+Castell%C3%B3%2C+and+J)
28. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In 1st International Conference on Learning Representations, ICLR 2013, Scottsdale, Arizona, USA, May 2-4, 2013, Workshop Track Proceedings. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Kai+Chen%2C+Greg+Corrado%2C+and+Jeffrey+Dean%2E+2013%2E+Efficient+estimation+of+word+representations+in+vector+space%2E+In+1st+International+Conference+on+Learning+Representations%2C+ICLR+2013%2C+Sco)
29. Minh Van Nguyen, Viet Dac Lai, Amir Pouran Ben Veyseh, and Thien Huu Nguyen. 2021. Trankit: A light-weight transformer-based toolkit for multilingual natural language processing. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System De… [find](https://scholar.google.com/scholar?q=Minh+Van+Nguyen%2C+Viet+Dac+Lai%2C+Amir+Pouran+Ben+Veyseh%2C+and+Thien+Huu+Nguyen%2E+2021%2E+Trankit%3A+A+light%2Dweight+transformer%2Dbased+toolkit+for+multilingual+natural+language+processing%2E+In+Proceedings+of+the) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2021-nguyen-trankit-light-weight-transformer-based/)
30. Martha Palmer, Rajesh Bhatt, Bhuvana Narasimhan, Owen Rambow, Dipti Misra Sharma, and Fei Xia. 2009. Hindi syntax: Annotating dependency, lexical predicate-argument structure, and phrase structure. In The 7th International Conference on Natural Language Processing, pages 14–17. [find](https://scholar.google.com/scholar?q=Martha+Palmer%2C+Rajesh+Bhatt%2C+Bhuvana+Narasimhan%2C+Owen+Rambow%2C+Dipti+Misra+Sharma%2C+and+Fei+Xia%2E+2009%2E+Hindi+syntax%3A+Annotating+dependency%2C+lexical+predicate%2Dargument+structure%2C+and+phrase+structure%2E+In)
31. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Te-jani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and… [find](https://scholar.google.com/scholar?q=Adam+Paszke%2C+Sam+Gross%2C+Francisco+Massa%2C+Adam+Lerer%2C+James+Bradbury%2C+Gregory+Chanan%2C+Trevor+Killeen%2C+Zeming+Lin%2C+Natalia+Gimelshein%2C+Luca+Antiga%2C+Alban+Desmaison%2C+Andreas+Kopf%2C+Edward+Yang%2C+Zachary+De)
32. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog… [find](https://scholar.google.com/scholar?q=Matthew+Peters%2C+Mark+Neumann%2C+Mohit+Iyyer%2C+Matt+Gardner%2C+Christopher+Clark%2C+Kenton+Lee%2C+and+Luke+Zettlemoyer%2E+2018%2E+Deep+contextualized+word+representations%2E+In+Proceedings+of+the+2018+Conference+of+t)
33. Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebastian Ruder, Kyunghyun Cho, and Iryna Gurevych. 2020a. AdapterHub: A framework for adapting transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations,… [find](https://scholar.google.com/scholar?q=Jonas+Pfeiffer%2C+Andreas+R%C3%BCckl%C3%A9%2C+Clifton+Poth%2C+Aishwarya+Kamath%2C+Ivan+Vulic%CC%81%2C+Sebastian+Ruder%2C+Kyunghyun+Cho%2C+and+Iryna+Gurevych%2E+2020a%2E+AdapterHub%3A+A+framework+for+adapting+transformers%2E+In+Proceeding)
34. Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Sebastian Ruder. 2020b. MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7654–7673, Online. Association for Computational… [find](https://scholar.google.com/scholar?q=Jonas+Pfeiffer%2C+Ivan+Vulic%CC%81%2C+Iryna+Gurevych%2C+and+Sebastian+Ruder%2E+2020b%2E+MAD%2DX%3A+An+Adapter%2DBased+Framework+for+Multi%2DTask+Cross%2DLingual+Transfer%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Met)
35. Lauma Pretkalnin , a, Laura Rituma, and Baiba Saulı̄te. 2018. Deriving enhanced universal dependencies from a hybrid dependency-constituency treebank. In International Conference on Text, Speech, and Dialogue, pages 95–105. Springer. [find](https://scholar.google.com/scholar?q=Lauma+Pretkalnin+%2C+a%2C+Laura+Rituma%2C+and+Baiba+Saul%C4%B1%CC%84te%2E+2018%2E+Deriving+enhanced+universal+dependencies+from+a+hybrid+dependency%2Dconstituency+treebank%2E+In+International+Conference+on+Text%2C+Speech%2C+and+)
36. Sampo Pyysalo, Jenna Kanerva, Antti Virtanen, and Filip Ginter. 2020. Wikibert models: deep transfer learning for many languages. arXiv preprint arXiv:2006.01538. [find](https://scholar.google.com/scholar?q=Sampo+Pyysalo%2C+Jenna+Kanerva%2C+Antti+Virtanen%2C+and+Filip+Ginter%2E+2020%2E+Wikibert+models%3A+deep+transfer+learning+for+many+languages%2E+arXiv+preprint+arXiv%3A2006%2E01538%2E)
37. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A Python natural language processing toolkit for many human languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. [find](https://scholar.google.com/scholar?q=Peng+Qi%2C+Yuhao+Zhang%2C+Yuhui+Zhang%2C+Jason+Bolton%2C+and+Christopher+D%2E+Manning%2E+2020%2E+Stanza%3A+A+Python+natural+language+processing+toolkit+for+many+human+languages%2E+In+Proceedings+of+the+58th+Annual+Meet) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2020-qi-stanza-python-natural-language-processing/)
38. Ali Safaya, Moutasem Abdullatif, and Deniz Yuret. 2020. Kuisail at semeval-2020 task 12: Bert-cnn for offensive speech identification in social media. [find](https://scholar.google.com/scholar?q=Ali+Safaya%2C+Moutasem+Abdullatif%2C+and+Deniz+Yuret%2E+2020%2E+Kuisail+at+semeval%2D2020+task+12%3A+Bert%2Dcnn+for+offensive+speech+identification+in+social+media%2E)
39. Sebastian Schuster and Christopher D. Manning. 2016. Enhanced English Universal Dependencies: An improved representation for natural language understanding tasks. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 2371–2378, Portorož, Slovenia.… [find](https://scholar.google.com/scholar?q=Sebastian+Schuster+and+Christopher+D%2E+Manning%2E+2016%2E+Enhanced+English+Universal+Dependencies%3A+An+improved+representation+for+natural+language+understanding+tasks%2E+In+Proceedings+of+the+Tenth+Internati) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2016-schuster-enhanced-english-universal-dependencies-improved/)
40. Tianze Shi, Igor Malioutov, and Ozan Irsoy. 2020. Semantic role labeling as syntactic dependency parsing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7551–7571, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Tianze+Shi%2C+Igor+Malioutov%2C+and+Ozan+Irsoy%2E+2020%2E+Semantic+role+labeling+as+syntactic+dependency+parsing%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMN) [our card](https://md.mova.elamur.ai/free/en/docs/papers/shi-2020-semantic-role-labeling-as-syntactic/)
41. Natalia Silveira, Timothy Dozat, Marie-Catherine de Marneffe, Samuel Bowman, Miriam Connor, John Bauer, and Christopher D. Manning. 2014. A gold standard dependency corpus for English. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014). [find](https://scholar.google.com/scholar?q=Natalia+Silveira%2C+Timothy+Dozat%2C+Marie%2DCatherine+de+Marneffe%2C+Samuel+Bowman%2C+Miriam+Connor%2C+John+Bauer%2C+and+Christopher+D%2E+Manning%2E+2014%2E+A+gold+standard+dependency+corpus+for+English%2E+In+Proceedings+)
42. Aaron Smith, Miryam de Lhoneux, Sara Stymne, and Joakim Nivre. 2018. An investigation of the interactions between pre-trained word embeddings, character models and POS tags in dependency parsing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2711–27… [find](https://scholar.google.com/scholar?q=Aaron+Smith%2C+Miryam+de+Lhoneux%2C+Sara+Stymne%2C+and+Joakim+Nivre%2E+2018%2E+An+investigation+of+the+interactions+between+pre%2Dtrained+word+embeddings%2C+character+models+and+POS+tags+in+dependency+parsing%2E+In+P)
43. Otakar Smrž, Viktor Bielický, Iveta Kouřilová, Jakub Kráčmar, Jan Hajič, and Petr Zemánek. 2008. Prague arabic dependency treebank: A word on the million words. In Proceedings of the Workshop on Arabic and Local Languages (LREC 2008), pages 16–23, Marrakech, Morocco. [find](https://scholar.google.com/scholar?q=Otakar+Smr%C5%BE%2C+Viktor+Bielick%C3%BD%2C+Iveta+Kour%CC%8Cilov%C3%A1%2C+Jakub+Kr%C3%A1c%CC%8Cmar%2C+Jan+Hajic%CC%8C%2C+and+Petr+Zem%C3%A1nek%2E+2008%2E+Prague+arabic+dependency+treebank%3A+A+word+on+the+million+words%2E+In+Proceedings+of+the+Workshop+on+Ar)
44. Milan Straka. 2018. UDPipe 2.0 prototype at CoNLL 2018 UD shared task. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 197–207, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Milan+Straka%2E+2018%2E+UDPipe+2%2E0+prototype+at+CoNLL+2018+UD+shared+task%2E+In+Proceedings+of+the+CoNLL+2018+Shared+Task%3A+Multilingual+Parsing+from+Raw+Text+to+Universal+Dependencies%2C+pages+197%E2%80%93207%2C+Brusse) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2018-straka-udpipe-2-0-prototype-conll/)
45. Milan Straka, Jana Straková, and Jan Hajič. 2019. Evaluating contextualized embeddings on 54 languages in pos tagging, lemmatization and dependency parsing. arXiv preprint arXiv:1908.07448. [find](https://scholar.google.com/scholar?q=Milan+Straka%2C+Jana+Strakov%C3%A1%2C+and+Jan+Hajic%CC%8C%2E+2019%2E+Evaluating+contextualized+embeddings+on+54+languages+in+pos+tagging%2C+lemmatization+and+dependency+parsing%2E+arXiv+preprint+arXiv%3A1908%2E07448%2E)
46. Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019a. BERT rediscovers the classical NLP pipeline. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4593– 4601, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ian+Tenney%2C+Dipanjan+Das%2C+and+Ellie+Pavlick%2E+2019a%2E+BERT+rediscovers+the+classical+NLP+pipeline%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+4593%E2%80%93)
47. Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Sam Bowman, Dipanjan Das, and Ellie Pavlick. 2019b. What do you learn from context? probing for sentence structure in contextualized word representations. In International Conference on Lea… [find](https://scholar.google.com/scholar?q=Ian+Tenney%2C+Patrick+Xia%2C+Berlin+Chen%2C+Alex+Wang%2C+Adam+Poliak%2C+R+Thomas+McCoy%2C+Najoung+Kim%2C+Benjamin+Van+Durme%2C+Sam+Bowman%2C+Dipanjan+Das%2C+and+Ellie+Pavlick%2E+2019b%2E+What+do+you+learn+from+context%3F+probi)
48. Ahmet Üstün, Arianna Bisazza, Gosse Bouma, and Gertjan van Noord. 2020. UDapter: Language adaptation for truly Universal Dependency parsing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2302–2315, Online. Association for Computational Lingu… [find](https://scholar.google.com/scholar?q=Ahmet+%C3%9Cst%C3%BCn%2C+Arianna+Bisazza%2C+Gosse+Bouma%2C+and+Gertjan+van+Noord%2E+2020%2E+UDapter%3A+Language+adaptation+for+truly+Universal+Dependency+parsing%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+)
49. Antti Virtanen, Jenna Kanerva, Rami Ilo, Jouni Luoma, Juhani Luotolahti, Tapio Salakoski, Filip Ginter, and Sampo Pyysalo. 2019. Multilingual is not enough: Bert for finnish. arXiv preprint arXiv:1912.07076. [find](https://scholar.google.com/scholar?q=Antti+Virtanen%2C+Jenna+Kanerva%2C+Rami+Ilo%2C+Jouni+Luoma%2C+Juhani+Luotolahti%2C+Tapio+Salakoski%2C+Filip+Ginter%2C+and+Sampo+Pyysalo%2E+2019%2E+Multilingual+is+not+enough%3A+Bert+for+finnish%2E+arXiv+preprint+arXiv%3A1912)
50. Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, and Edouard Grave. 2020. CCNet: Extracting high quality monolingual datasets from web crawl data. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 4003–4012, Marsei… [find](https://scholar.google.com/scholar?q=Guillaume+Wenzek%2C+Marie%2DAnne+Lachaux%2C+Alexis+Conneau%2C+Vishrav+Chaudhary%2C+Francisco+Guzm%C3%A1n%2C+Armand+Joulin%2C+and+Edouard+Grave%2E+2020%2E+CCNet%3A+Extracting+high+quality+monolingual+datasets+from+web+crawl+da)
51. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtow-icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lh… [find](https://scholar.google.com/scholar?q=Thomas+Wolf%2C+Lysandre+Debut%2C+Victor+Sanh%2C+Julien+Chaumond%2C+Clement+Delangue%2C+Anthony+Moi%2C+Pier%2Dric+Cistac%2C+Tim+Rault%2C+R%C3%A9mi+Louf%2C+Morgan+Funtow%2Dicz%2C+Joe+Davison%2C+Sam+Shleifer%2C+Patrick+von+Platen%2C+Clara)
52. Juntao Yu, Bernd Bohnet, and Massimo Poesio. 2020. Named entity recognition as dependency parsing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6470– 6476, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Juntao+Yu%2C+Bernd+Bohnet%2C+and+Massimo+Poesio%2E+2020%2E+Named+entity+recognition+as+dependency+parsing%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+647)
53. Daniel Zeman, Jan Hajič, Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre, and Slav Petrov. 2018. [find](https://scholar.google.com/scholar?q=Daniel+Zeman%2C+Jan+Hajic%CC%8C%2C+Martin+Popel%2C+Martin+Potthast%2C+Milan+Straka%2C+Filip+Ginter%2C+Joakim+Nivre%2C+and+Slav+Petrov%2E+2018%2E)
54. CoNLL 2018 shared task: Multilingual parsing from raw text to Universal Dependencies. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 1–21, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=CoNLL+2018+shared+task%3A+Multilingual+parsing+from+raw+text+to+Universal+Dependencies%2E+In+Proceedings+of+the+CoNLL+2018+Shared+Task%3A+Multilingual+Parsing+from+Raw+Text+to+Universal+Dependencies%2C+pages+) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2018-zeman-conll-2018-shared-task-multilingual/)
55. Daniel Zeman, Martin Popel, Milan Straka, Jan Hajič, Joakim Nivre, Filip Ginter, Juhani Luotolahti, Sampo Pyysalo, Slav Petrov, Martin Potthast, Francis Tyers, Elena Badmaeva, Memduh Gokirmak, Anna Nedoluzhko, Silvie Cinková, Jan Hajič jr., Jaroslava Hlaváčová, Václava Kettnerová, Zdeňka Urešová… [find](https://scholar.google.com/scholar?q=Daniel+Zeman%2C+Martin+Popel%2C+Milan+Straka%2C+Jan+Hajic%CC%8C%2C+Joakim+Nivre%2C+Filip+Ginter%2C+Juhani+Luotolahti%2C+Sampo+Pyysalo%2C+Slav+Petrov%2C+Martin+Potthast%2C+Francis+Tyers%2C+Elena+Badmaeva%2C+Memduh+Gokirmak%2C+Anna+N)
56. CoNLL 2017 shared task: Multilingual parsing from raw text to Universal Dependencies. In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 1–19, Vancouver, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=CoNLL+2017+shared+task%3A+Multilingual+parsing+from+raw+text+to+Universal+Dependencies%2E+In+Proceedings+of+the+CoNLL+2017+Shared+Task%3A+Multilingual+Parsing+from+Raw+Text+to+Universal+Dependencies%2C+pages+) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2017-zeman-conll-2017-shared-task-multilingual/)