[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/aclbulk-2010-nivre-evaluation-dependency-parsers-unbounded-dependencies/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / Evaluation of Dependency Parsers on Unbounded Dependencies

# Evaluation of Dependency Parsers on Unbounded Dependencies

**Authors:** [Joakim Nivre](https://md.mova.elamur.ai/free/en/docs/papers/authors/joakim-nivre/), [Laura Rimell](https://md.mova.elamur.ai/free/en/docs/papers/authors/laura-rimell/), [Ryan McDonald](https://md.mova.elamur.ai/free/en/docs/papers/authors/ryan-mcdonald/), [Carlos Gómez-Rodríguez](https://md.mova.elamur.ai/free/en/docs/papers/authors/carlos-gomez-rodriguez/) · **Year:** 2010 · **Venue:** Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010) **Link:** [aclanthology.org/C10-1094](https://aclanthology.org/C10-1094/) · **Licence:** [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/) — summary only

[**57** citations](https://openalex.org/works?filter=cites:W2157889740) openalex · 2026-10-18[**28** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/aclbulk-2010-nivre-evaluation-dependency-parsers-unbounded-dependencies/) of the 140673 papers whose bibliographies we read[Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010)](https://aclanthology.org/volumes/C10-1/)[licence](https://creativecommons.org/licenses/by-nc-sa/3.0/)

- [Publisher's page](https://aclanthology.org/C10-1094/)
- [PDF](https://aclanthology.org/C10-1094.pdf)
- [OpenAlex](https://openalex.org/W2157889740)

**Authors (4)**

1. [Joakim Nivre](https://openalex.org/A5077063699) h 60 · 16290 citations · 335 works
2. [Laura Rimell](https://openalex.org/A5059804251) h 18 · 2350 citations · 72 works
3. [Ryan McDonald](https://openalex.org/A5022489655) h 48 · 16372 citations · 113 works
4. Carlos Gómez-Rodríguez

Citations and author metrics from openalex (2026-10-18); authors with the same name are not always told apart.

## What it is about

Unbounded dependencies, such as an object that is extracted from a relative clause, are a standard test of grammar-based parsers, but had not been applied to dependency parsers. The authors train MSTParser and MaltParser on the Penn Treebank converted to Stanford dependencies, and add a simple post-processor that extracts long-range arcs from the basic tree. Their test follows Rimell et al. (2009). With the post-processor, the two parsers recover unbounded dependencies about 50% of the time, slightly below two grammar-based parsers designed for this. Recall varies by construction, from about 65% down to 1%.

## Ideas

- I1. `en/src/parse.rs` plus a post-processor in `prag`: add long-range arcs for object relatives and reduced relatives on the `en` tree, and measure the recall of these arcs on a 100-sentence dev set of relatives and questions, before and after, two runs each.
- I2. `gparse`: the biaffine and Eisner model may recover non-local arcs that the arc-standard parser cannot; run the same per-construction recall for `en` and `gparse` on the same dev set and report both.
- I3. `en parse-eval`: report recall per construction of unbounded arcs next to LAS, so that an overall gain which hides a loss on relatives is visible.

## For Mova

Touches [25. Relative clauses](https://md.mova.elamur.ai/free/en/docs/languages/en/25-relative-clauses/), [23. Complementation](https://md.mova.elamur.ai/free/en/docs/languages/en/23-complementation/) and [26. Questions](https://md.mova.elamur.ai/free/en/docs/languages/en/26-questions/). The `prag` and `world` layers need these arcs to link a relative clause to its antecedent. Measure I3 first, because it shows where the existing `en` output already fails.

Topics: [Dependency parsing](https://md.mova.elamur.ai/free/en/docs/papers/topics/dependency-parsing/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work.

```
@inproceedings{nivre2010evaluation,
  title = {{Evaluation of Dependency Parsers on Unbounded Dependencies}},
  author = {Nivre, Joakim and Rimell, Laura and McDonald, Ryan and Gómez-Rodríguez, Carlos},
  booktitle = {Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010)},
  year = {2010},
  url = {https://aclanthology.org/C10-1094/}
}
```

- [← previous "cba to check the spelling": Investigating Parser Performance on Discussion Forum Posts](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2010-foster-cba-check-spelling-investigating-parser/)
- [next → Multiword Expressions in the wild? The mwetoolkit comes in handy](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2010-ramisch-multiword-expressions-wild-mwetoolkit-comes/)
- [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/)

## References

28 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Black, E., S. Abney, D. Flickinger, C. Gdaniec, R. Grishman, P. Harrison, D. Hindle, R. Ingria, F. Jelinek, J. Klavans, M. Liberman, S. Roukos, B. Santorini, and T. Strzalkowski. 1991. A procedure for quantitatively comparing the syntactic coverage of English grammars. In Proceedings of 4th DARPA Workshop, 306–311. [find](https://scholar.google.com/scholar?q=Black%2C+E%2E%2C+S%2E+Abney%2C+D%2E+Flickinger%2C+C%2E+Gdaniec%2C+R%2E+Grishman%2C+P%2E+Harrison%2C+D%2E+Hindle%2C+R%2E+Ingria%2C+F%2E+Jelinek%2C+J%2E+Klavans%2C+M%2E+Liberman%2C+S%2E+Roukos%2C+B%2E+Santorini%2C+and+T%2E+Strzalkowski%2E+1991%2E+A+procedure+for)
2. Briscoe, T., J. Carroll, and R. Watson. 2006. The second release of the RASP system. In Proceedings of the COLING/ACL 2006 Interactive Presentation Sessions, 77–80. [find](https://scholar.google.com/scholar?q=Briscoe%2C+T%2E%2C+J%2E+Carroll%2C+and+R%2E+Watson%2E+2006%2E+The+second+release+of+the+RASP+system%2E+In+Proceedings+of+the+COLING%2FACL+2006+Interactive+Presentation+Sessions%2C+77%E2%80%9380%2E)
3. Buchholz, S. and E. Marsi. 2006. CoNLL-X shared task on multilingual dependency parsing. In Proceedings of CoNLL, 149–164. [find](https://scholar.google.com/scholar?q=Buchholz%2C+S%2E+and+E%2E+Marsi%2E+2006%2E+CoNLL%2DX+shared+task+on+multilingual+dependency+parsing%2E+In+Proceedings+of+CoNLL%2C+149%E2%80%93164%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/parsers-2006-buchholz-conll-x-shared-task-multilingual/)
4. Cahill, A., M. Burke, R. O’Donovan, J. Van Genabith, and A. Way. 2004. Long-distance dependency resolution in automatically acquired wide-coverage PCFG-based LFG approximations. In Proceedings of ACL, 320–327. [find](https://scholar.google.com/scholar?q=Cahill%2C+A%2E%2C+M%2E+Burke%2C+R%2E+O%E2%80%99Donovan%2C+J%2E+Van+Genabith%2C+and+A%2E+Way%2E+2004%2E+Long%2Ddistance+dependency+resolution+in+automatically+acquired+wide%2Dcoverage+PCFG%2Dbased+LFG+approximations%2E+In+Proceedings+of+ACL%2C)
5. Carreras, X., M. Collins, and T. Koo. 2008. TAG, dynamic programming, and the perceptron for efficient, feature-rich parsing. In Proceedings of CoNLL, 9–16. [find](https://scholar.google.com/scholar?q=Carreras%2C+X%2E%2C+M%2E+Collins%2C+and+T%2E+Koo%2E+2008%2E+TAG%2C+dynamic+programming%2C+and+the+perceptron+for+efficient%2C+feature%2Drich+parsing%2E+In+Proceedings+of+CoNLL%2C+9%E2%80%9316%2E)
6. Charniak, E. and M. Johnson. 2005. Coarse-to-fine nbest parsing and MaxEnt discriminative reranking. In Proceedings of ACL, 173–180. [find](https://scholar.google.com/scholar?q=Charniak%2C+E%2E+and+M%2E+Johnson%2E+2005%2E+Coarse%2Dto%2Dfine+nbest+parsing+and+MaxEnt+discriminative+reranking%2E+In+Proceedings+of+ACL%2C+173%E2%80%93180%2E)
7. Clark, S. and J. R. Curran. 2007. Wide-coverage efficient statistical parsing with CCG and log-linear models. Computational Linguistics, 33:493–552. [find](https://scholar.google.com/scholar?q=Clark%2C+S%2E+and+J%2E+R%2E+Curran%2E+2007%2E+Wide%2Dcoverage+efficient+statistical+parsing+with+CCG+and+log%2Dlinear+models%2E+Computational+Linguistics%2C+33%3A493%E2%80%93552%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/nlpgap-2007-clark-wide-coverage-efficient-statistical-parsing/)
8. de Marneffe, M.-C., B. MacCartney, and C. D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In Proceedings of LREC. [find](https://scholar.google.com/scholar?q=de+Marneffe%2C+M%2E%2DC%2E%2C+B%2E+MacCartney%2C+and+C%2E+D%2E+Manning%2E+2006%2E+Generating+typed+dependency+parses+from+phrase+structure+parses%2E+In+Proceedings+of+LREC%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/map-2006-marneffe-generating-typed-dependency-parses-phrase/)
9. Giménez, J. and L. Màrquez. 2004. SVMTool: A general POS tagger generator based on support vector machines. In Proceedings of LREC. [find](https://scholar.google.com/scholar?q=Gime%CC%81nez%2C+J%2E+and+L%2E+Ma%CC%80rquez%2E+2004%2E+SVMTool%3A+A+general+POS+tagger+generator+based+on+support+vector+machines%2E+In+Proceedings+of+LREC%2E)
10. Hall, J., J. Nivre, and J. Nilsson. 2006. Discriminative classifiers for deterministic dependency parsing. In Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions, 316–323. [find](https://scholar.google.com/scholar?q=Hall%2C+J%2E%2C+J%2E+Nivre%2C+and+J%2E+Nilsson%2E+2006%2E+Discriminative+classifiers+for+deterministic+dependency+parsing%2E+In+Proceedings+of+the+COLING%2FACL+2006+Main+Conference+Poster+Sessions%2C+316%E2%80%93323%2E)
11. Hall, J., J. Nilsson, J. Nivre, G. Eryiğit, B. Megyesi, M. Nilsson, and M. Saers. 2007. Single malt or blended? A study in multilingual parser optimization. In Proceedings of the CoNLL Shared Task. [find](https://scholar.google.com/scholar?q=Hall%2C+J%2E%2C+J%2E+Nilsson%2C+J%2E+Nivre%2C+G%2E+Eryig%CC%86it%2C+B%2E+Megyesi%2C+M%2E+Nilsson%2C+and+M%2E+Saers%2E+2007%2E+Single+malt+or+blended%3F+A+study+in+multilingual+parser+optimization%2E+In+Proceedings+of+the+CoNLL+Shared+Task%2E)
12. Huang, L. 2008. Forest reranking: Discriminative parsing with non-local features. In Proceedings of ACL, 586–594. [find](https://scholar.google.com/scholar?q=Huang%2C+L%2E+2008%2E+Forest+reranking%3A+Discriminative+parsing+with+non%2Dlocal+features%2E+In+Proceedings+of+ACL%2C+586%E2%80%93594%2E)
13. Judge, J., A. Cahill, and J. van Genabith. 2006. QuestionBank: Creating a corpus of parse-annotated questions. In Proceedings of COLING-ACL, 497– 504. [find](https://scholar.google.com/scholar?q=Judge%2C+J%2E%2C+A%2E+Cahill%2C+and+J%2E+van+Genabith%2E+2006%2E+QuestionBank%3A+Creating+a+corpus+of+parse%2Dannotated+questions%2E+In+Proceedings+of+COLING%2DACL%2C+497%E2%80%93+504%2E)
14. Klein, D. and C. D. Manning. 2003. Accurate unlexicalized parsing. In Proceedings of ACL, 423–430. [find](https://scholar.google.com/scholar?q=Klein%2C+D%2E+and+C%2E+D%2E+Manning%2E+2003%2E+Accurate+unlexicalized+parsing%2E+In+Proceedings+of+ACL%2C+423%E2%80%93430%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2003-klein-accurate-unlexicalized-parsing/)
15. Kübler, S., R. McDonald, and J. Nivre. 2008. Dependency Parsing. Morgan and Claypool. [find](https://scholar.google.com/scholar?q=Ku%CC%88bler%2C+S%2E%2C+R%2E+McDonald%2C+and+J%2E+Nivre%2E+2008%2E+Dependency+Parsing%2E+Morgan+and+Claypool%2E)
16. McDonald, R. and J. Nivre. 2007. Characterizing the errors of data-driven dependency parsing models. In Proceedings of EMNLP-CoNLL, 122–131. [find](https://scholar.google.com/scholar?q=McDonald%2C+R%2E+and+J%2E+Nivre%2E+2007%2E+Characterizing+the+errors+of+data%2Ddriven+dependency+parsing+models%2E+In+Proceedings+of+EMNLP%2DCoNLL%2C+122%E2%80%93131%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/missing4-2007-mcdonald-characterizing-errors-data-driven-dependency/)
17. McDonald, R. and F. Pereira. 2006. Online learning of approximate dependency parsing algorithms. In Proceedings of EACL, 81–88. [find](https://scholar.google.com/scholar?q=McDonald%2C+R%2E+and+F%2E+Pereira%2E+2006%2E+Online+learning+of+approximate+dependency+parsing+algorithms%2E+In+Proceedings+of+EACL%2C+81%E2%80%9388%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/continual-2006-mcdonald-online-learning-approximate-dependency-parsing/)
18. McDonald, R. and G. Satta. 2007. On the complexity of non-projective data-driven dependency parsing. In Proceedings of IWPT, 122–131. [find](https://scholar.google.com/scholar?q=McDonald%2C+R%2E+and+G%2E+Satta%2E+2007%2E+On+the+complexity+of+non%2Dprojective+data%2Ddriven+dependency+parsing%2E+In+Proceedings+of+IWPT%2C+122%E2%80%93131%2E)
19. McDonald, R., K. Crammer, and F. Pereira. 2005. Online large-margin training of dependency parsers. In Proceedings of ACL, 91–98. [find](https://scholar.google.com/scholar?q=McDonald%2C+R%2E%2C+K%2E+Crammer%2C+and+F%2E+Pereira%2E+2005%2E+Online+large%2Dmargin+training+of+dependency+parsers%2E+In+Proceedings+of+ACL%2C+91%E2%80%9398%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/narrative-2005-mcdonald-online-large-margin-training-dependency/)
20. McDonald, R., K. Lerman, and F. Pereira. 2006. Multilingual dependency analysis with a two-stage discriminative parser. In Proceedings of CoNLL, 216– 220. [find](https://scholar.google.com/scholar?q=McDonald%2C+R%2E%2C+K%2E+Lerman%2C+and+F%2E+Pereira%2E+2006%2E+Multilingual+dependency+analysis+with+a+two%2Dstage+discriminative+parser%2E+In+Proceedings+of+CoNLL%2C+216%E2%80%93+220%2E)
21. McDonald, R.. 2006. Discriminative Learning and Spanning Tree Algorithms for Dependency Parsing. Ph.D. thesis, University of Pennsylvania. [find](https://scholar.google.com/scholar?q=McDonald%2C+R%2E%2E+2006%2E+Discriminative+Learning+and+Spanning+Tree+Algorithms+for+Dependency+Parsing%2E+Ph%2ED%2E+thesis%2C+University+of+Pennsylvania%2E)
22. Miyao, Y. and J. Tsujii. 2005. Probabilistic disambiguation models for wide-coverage HPSG parsing. In Proceedings of ACL, 83–90. [find](https://scholar.google.com/scholar?q=Miyao%2C+Y%2E+and+J%2E+Tsujii%2E+2005%2E+Probabilistic+disambiguation+models+for+wide%2Dcoverage+HPSG+parsing%2E+In+Proceedings+of+ACL%2C+83%E2%80%9390%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2005-miyao-probabilistic-disambiguation-models-wide-coverage/)
23. Nivre, J., J. Hall, and J. Nilsson. 2006a. MaltParser: A data-driven parser-generator for dependency parsing. In Proceedings of LREC, 2216–2219. [find](https://scholar.google.com/scholar?q=Nivre%2C+J%2E%2C+J%2E+Hall%2C+and+J%2E+Nilsson%2E+2006a%2E+MaltParser%3A+A+data%2Ddriven+parser%2Dgenerator+for+dependency+parsing%2E+In+Proceedings+of+LREC%2C+2216%E2%80%932219%2E)
24. Nivre, J., J. Hall, J. Nilsson, G. Eryiğit, and S. Marinov. 2006b. Labeled pseudo-projective dependency parsing with support vector machines. In Proceedings of CoNLL, 221–225. [find](https://scholar.google.com/scholar?q=Nivre%2C+J%2E%2C+J%2E+Hall%2C+J%2E+Nilsson%2C+G%2E+Eryig%CC%86it%2C+and+S%2E+Marinov%2E+2006b%2E+Labeled+pseudo%2Dprojective+dependency+parsing+with+support+vector+machines%2E+In+Proceedings+of+CoNLL%2C+221%E2%80%93225%2E)
25. Nivre, J., J. Hall, J. Nilsson, A. Chanev, G. Eryiǧit, S. Kübler, S. Marinov, and E. Marsi. 2007. Maltparser: A language-independent system for data-driven dependency parsing. Natural Language Engineering, 13:95–135. [find](https://scholar.google.com/scholar?q=Nivre%2C+J%2E%2C+J%2E+Hall%2C+J%2E+Nilsson%2C+A%2E+Chanev%2C+G%2E+Eryig%CC%8Cit%2C+S%2E+Ku%CC%88bler%2C+S%2E+Marinov%2C+and+E%2E+Marsi%2E+2007%2E+Maltparser%3A+A+language%2Dindependent+system+for+data%2Ddriven+dependency+parsing%2E+Natural+Language+Engin)
26. Nivre, J. 2009. Non-projective dependency parsing in expected linear time. In Proceedings of ACL-IJCNLP, 351–359. [find](https://scholar.google.com/scholar?q=Nivre%2C+J%2E+2009%2E+Non%2Dprojective+dependency+parsing+in+expected+linear+time%2E+In+Proceedings+of+ACL%2DIJCNLP%2C+351%E2%80%93359%2E)
27. Rimell, L., S. Clark, and M. Steedman. 2009. Unbounded dependency recovery for parser evaluation. In Proceedings EMNLP, 813–821. [find](https://scholar.google.com/scholar?q=Rimell%2C+L%2E%2C+S%2E+Clark%2C+and+M%2E+Steedman%2E+2009%2E+Unbounded+dependency+recovery+for+parser+evaluation%2E+In+Proceedings+EMNLP%2C+813%E2%80%93821%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2009-rimell-unbounded-dependency-recovery-parser-evaluation/)
28. Sagae, K. and A. Lavie. 2006. Parser combination by reparsing. In Proceedings of NAACL HLT: Short Papers, 129–132. [find](https://scholar.google.com/scholar?q=Sagae%2C+K%2E+and+A%2E+Lavie%2E+2006%2E+Parser+combination+by+reparsing%2E+In+Proceedings+of+NAACL+HLT%3A+Short+Papers%2C+129%E2%80%93132%2E)