[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/dickinson-2009-correcting-dependency-annotation-errors/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / Correcting Dependency Annotation Errors

# Correcting Dependency Annotation Errors

**Authors:** [Markus Dickinson](https://md.mova.elamur.ai/free/en/docs/papers/authors/markus-dickinson/) · **Year:** 2009 · **Venue:** Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009) **Link:** [aclanthology.org/E09-1023](https://aclanthology.org/E09-1023/) (ACL Anthology E09-1023) **License of the paper:** [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/) (ACL Anthology, pre-2016) — [creativecommons.org/licenses/by-nc-sa/3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/)

[**7** citations](https://openalex.org/works?filter=cites:W2027292098) openalex · 2026-10-19[**1** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/dickinson-2009-correcting-dependency-annotation-errors/) of the 140673 papers whose bibliographies we read[Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009)](https://aclanthology.org/volumes/E09-1/)[licence](https://creativecommons.org/licenses/by-nc-sa/3.0/)

- [Publisher's page](https://aclanthology.org/E09-1023/)
- [PDF](https://aclanthology.org/E09-1023.pdf)
- [DOI 10.3115/1609067.1609088](https://doi.org/10.3115/1609067.1609088)
- [OpenAlex](https://openalex.org/W2027292098)

**Authors (1)**

1. [Dickinson, Markus](https://openalex.org/A5063365523) [Semantic Scholar](https://www.semanticscholar.org/author/144503727) · h 14 · 819 citations · 103 works

Citations and author metrics from openalex (2026-10-19); authors with the same name are not always told apart.

## Summary

The paper moves from detecting errors in dependency treebank annotation to correcting them automatically. It explains why this is hard: erroneous annotations often recur identically in similar contexts, so a model trained on the corpus easily learns the noise. The data is the written part of the Swedish Talbanken05 treebank, with correction candidates taken from variation n-grams found by earlier detection work. For each word pair the model predicts a dependency label together with which word is the head, or a special label when there is no relation; classification uses memory-based learning. Simple combinations of local features perform poorly, so the author takes ambiguity classes from strict lexical models and passes them to more flexible POS-based models, as output constraints, as extra features, or both. The best configuration labels about three quarters of the problematic positions correctly, clearly above the no-change baseline. The idea of one model refining another is presented as applicable to other annotation layers and to dependency parsing.

## How Mova uses it

- en/seeds/errors/consistency-detection-methods.md: a seed card surveying automatic ways to find annotation errors; this paper is cited for method 1 ("variation nuclei" — the same word pair in the same context with different relations signals an error) and for the result that such positions can be automatically corrected (76.7% vs 70.1% unchanged, as reported in the paper).
- The card feeds the expert-system principle that a rule which fires often on a gold treebank is either wrong or is catching errors in the gold standard, and the noise-robust training in Mova's taggers that turns repeatedly-wrong labels into suspicions rather than facts.

## Effectiveness in Mova

Background reading; not measured separately. No Dickinson-style correction classifier is implemented in Mova.

---

👨‍🔬💥

Topics: [Treebanks](https://md.mova.elamur.ai/free/en/docs/papers/topics/treebanks/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.3115/1609067.1609088](https://doi.org/10.3115/1609067.1609088)

```
@inproceedings{dickinson2009correcting,
  title = {{Correcting Dependency Annotation Errors}},
  author = {Dickinson, Markus},
  booktitle = {Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009)},
  year = {2009},
  doi = {10.3115/1609067.1609088},
  url = {https://doi.org/10.3115/1609067.1609088}
}
```

- [← previous Universal Dependencies](https://md.mova.elamur.ai/free/en/docs/papers/demarneffe-2021-universal-dependencies/)
- [next → On Grammaticality in the Syntactic Annotation of Learner Language](https://md.mova.elamur.ai/free/en/docs/papers/dickinson-2015-grammaticality-syntactic-annotation-learner-language/)
- [↑ back to papers](https://md.mova.elamur.ai/free/en/docs/papers/)

## References

26 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Boyd, Adriane, Markus Dickinson and Detmar Meurers (2008). On Detecting Errors in Dependency Treebanks. Research on Language and Computation 6(2), 113–137. [find](https://scholar.google.com/scholar?q=Boyd%2C+Adriane%2C+Markus+Dickinson+and+Detmar+Meurers+%282008%29%2E+On+Detecting+Errors+in+Dependency+Treebanks%2E+Research+on+Language+and+Computation+6%282%29%2C+113%E2%80%93137%2E)
2. Canisius, Sander, Toine Bogers, Antal van den Bosch, Jeroen Geertzen and Erik Tjong Kim Sang (2006). Dependency parsing by inference over high-recall dependency predictions. In Proceedings of CoNLL-X. New York. [find](https://scholar.google.com/scholar?q=Canisius%2C+Sander%2C+Toine+Bogers%2C+Antal+van+den+Bosch%2C+Jeroen+Geertzen+and+Erik+Tjong+Kim+Sang+%282006%29%2E+Dependency+parsing+by+inference+over+high%2Drecall+dependency+predictions%2E+In+Proceedings+of+CoNLL%2DX%2E)
3. Chen, Wenliang, Youzheng Wu and Hitoshi Isahara (2008). Learning Reliable Information for Dependency Parsing Adaptation. In Proceedings of Coling 2008. Manchester. [find](https://scholar.google.com/scholar?q=Chen%2C+Wenliang%2C+Youzheng+Wu+and+Hitoshi+Isahara+%282008%29%2E+Learning+Reliable+Information+for+Dependency+Parsing+Adaptation%2E+In+Proceedings+of+Coling+2008%2E+Manchester%2E)
4. Daelemans, Walter (1996). Abstraction Considered Harmful: Lazy Learning of Language Processing. In Proceedings of the 6th Belgian-Dutch Conference on Machine Learning. Maastricht, The Netherlands. [find](https://scholar.google.com/scholar?q=Daelemans%2C+Walter+%281996%29%2E+Abstraction+Considered+Harmful%3A+Lazy+Learning+of+Language+Processing%2E+In+Proceedings+of+the+6th+Belgian%2DDutch+Conference+on+Machine+Learning%2E+Maastricht%2C+The+Netherlands%2E)
5. Daelemans, Walter, Jakub Zavrel, Ko Van der Sloot and Antal Van den Bosch (2007). TiMBL: Tilburg Memory Based Learner, version 6.1, Reference Guide. Tech. rep., ILK Research Group. ILK Research Group Technical Report Series no. 07-07. [find](https://scholar.google.com/scholar?q=Daelemans%2C+Walter%2C+Jakub+Zavrel%2C+Ko+Van+der+Sloot+and+Antal+Van+den+Bosch+%282007%29%2E+TiMBL%3A+Tilburg+Memory+Based+Learner%2C+version+6%2E1%2C+Reference+Guide%2E+Tech%2E+rep%2E%2C+ILK+Research+Group%2E+ILK+Research+Group+)
6. Dickinson, Markus (2006). From Detecting Errors to Automatically Correcting Them. In Proceedings of EACL-06. Trento, Italy. [find](https://scholar.google.com/scholar?q=Dickinson%2C+Markus+%282006%29%2E+From+Detecting+Errors+to+Automatically+Correcting+Them%2E+In+Proceedings+of+EACL%2D06%2E+Trento%2C+Italy%2E)
7. Dickinson, Markus (2007). Determining Ambiguity Classes for Part-of-Speech Tagging. In Proceedings of RANLP-07. Borovets, Bulgaria. [find](https://scholar.google.com/scholar?q=Dickinson%2C+Markus+%282007%29%2E+Determining+Ambiguity+Classes+for+Part%2Dof%2DSpeech+Tagging%2E+In+Proceedings+of+RANLP%2D07%2E+Borovets%2C+Bulgaria%2E)
8. Dickinson, Markus and W. Detmar Meurers (2003). Detecting Inconsistencies in Treebanks. In Proceedings of TLT-03. Växjö, Sweden. [find](https://scholar.google.com/scholar?q=Dickinson%2C+Markus+and+W%2E+Detmar+Meurers+%282003%29%2E+Detecting+Inconsistencies+in+Treebanks%2E+In+Proceedings+of+TLT%2D03%2E+Va%CC%88xjo%CC%88%2C+Sweden%2E)
9. Dickinson, Markus and W. Detmar Meurers (2005). Detecting Errors in Discontinuous Structural Annotation. In Proceedings of ACL-05. [find](https://scholar.google.com/scholar?q=Dickinson%2C+Markus+and+W%2E+Detmar+Meurers+%282005%29%2E+Detecting+Errors+in+Discontinuous+Structural+Annotation%2E+In+Proceedings+of+ACL%2D05%2E)
10. Einarsson, Jan (1976). Talbankens skrift-sprøakskonkordans. Tech. rep., Lund University, Dept. of Scandinavian Languages. [find](https://scholar.google.com/scholar?q=Einarsson%2C+Jan+%281976%29%2E+Talbankens+skrift%2Dspr%C3%B8akskonkordans%2E+Tech%2E+rep%2E%2C+Lund+University%2C+Dept%2E+of+Scandinavian+Languages%2E)
11. Habash, Nizar, Ryan Gabbard, Owen Rambow, Seth Kulick and Mitch Marcus (2007). Determining Case in Arabic: Learning Complex Linguistic Behavior Requires Complex Linguistic Features. In Proceedings of EMNLP-07. [find](https://scholar.google.com/scholar?q=Habash%2C+Nizar%2C+Ryan+Gabbard%2C+Owen+Rambow%2C+Seth+Kulick+and+Mitch+Marcus+%282007%29%2E+Determining+Case+in+Arabic%3A+Learning+Complex+Linguistic+Behavior+Requires+Complex+Linguistic+Features%2E+In+Proceedings+of+)
12. Hogan, Deirdre (2007). Coordinate Noun Phrase Disambiguation in a Generative Parsing Model. In Proceedings of ACL-07. Prague. [find](https://scholar.google.com/scholar?q=Hogan%2C+Deirdre+%282007%29%2E+Coordinate+Noun+Phrase+Disambiguation+in+a+Generative+Parsing+Model%2E+In+Proceedings+of+ACL%2D07%2E+Prague%2E)
13. Klein, Dan and Christopher D. Manning (2002). A Generative Constituent-Context Model for Improved Grammar Induction. In Proceedings of ACL-02. Philadelphia, PA. [find](https://scholar.google.com/scholar?q=Klein%2C+Dan+and+Christopher+D%2E+Manning+%282002%29%2E+A+Generative+Constituent%2DContext+Model+for+Improved+Grammar+Induction%2E+In+Proceedings+of+ACL%2D02%2E+Philadelphia%2C+PA%2E)
14. McDonald, Ryan, Kevin Lerman and Fernando Pereira (2006). Multilingual Dependency Analysis with a Two-Stage Discriminative Parser. In Proceedings of CoNLL-X. New York City. [find](https://scholar.google.com/scholar?q=McDonald%2C+Ryan%2C+Kevin+Lerman+and+Fernando+Pereira+%282006%29%2E+Multilingual+Dependency+Analysis+with+a+Two%2DStage+Discriminative+Parser%2E+In+Proceedings+of+CoNLL%2DX%2E+New+York+City%2E)
15. McDonald, Ryan and Joakim Nivre (2007). Characterizing the Errors of Data-Driven Dependency Parsing Models. In Proceedings of EMNLP-CoNLL-07. Prague, pp. 122–131. [find](https://scholar.google.com/scholar?q=McDonald%2C+Ryan+and+Joakim+Nivre+%282007%29%2E+Characterizing+the+Errors+of+Data%2DDriven+Dependency+Parsing+Models%2E+In+Proceedings+of+EMNLP%2DCoNLL%2D07%2E+Prague%2C+pp%2E+122%E2%80%93131%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/missing4-2007-mcdonald-characterizing-errors-data-driven-dependency/)
16. McDonald, Ryan and Fernando Pereira (2006). Online learning of approximate dependency parsing algorithms. In Proceedings of EACL-06. Trento. [find](https://scholar.google.com/scholar?q=McDonald%2C+Ryan+and+Fernando+Pereira+%282006%29%2E+Online+learning+of+approximate+dependency+parsing+algorithms%2E+In+Proceedings+of+EACL%2D06%2E+Trento%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/continual-2006-mcdonald-online-learning-approximate-dependency-parsing/)
17. Mintz, Toben H. (2006). Finding the verbs: distributional cues to categories available to young learners. In K. Hirsh-Pasek and R. M. Golinkoff (eds.), Action Meets Word: How Children Learn Verbs, New York: Oxford University Press, pp. 31–63. [find](https://scholar.google.com/scholar?q=Mintz%2C+Toben+H%2E+%282006%29%2E+Finding+the+verbs%3A+distributional+cues+to+categories+available+to+young+learners%2E+In+K%2E+Hirsh%2DPasek+and+R%2E+M%2E+Golinkoff+%28eds%2E%29%2C+Action+Meets+Word%3A+How+Children+Learn+Verbs%2C+New)
18. Nivre, Joakim (2006). Inductive Dependency Parsing. Berlin: Springer. [find](https://scholar.google.com/scholar?q=Nivre%2C+Joakim+%282006%29%2E+Inductive+Dependency+Parsing%2E+Berlin%3A+Springer%2E)
19. Nivre, Joakim, Johan Hall, Jens Nilsson, Atanas Chanev, Gulsen Eryigit, Sandra Kubler, Svetoslav Marinov and Erwin Marsi (2007). Malt-Parser: A language-independent system for data-driven dependency parsing. Natural Language Engineering 13(2), 95–135. [find](https://scholar.google.com/scholar?q=Nivre%2C+Joakim%2C+Johan+Hall%2C+Jens+Nilsson%2C+Atanas+Chanev%2C+Gulsen+Eryigit%2C+Sandra+Kubler%2C+Svetoslav+Marinov+and+Erwin+Marsi+%282007%29%2E+Malt%2DParser%3A+A+language%2Dindependent+system+for+data%2Ddriven+dependency+p)
20. Nivre, Joakim and Ryan McDonald (2008). Integrating Graph-Based and Transition-Based Dependency Parsers. In Proceedings of ACL-08: HLT. Columbus, OH. [find](https://scholar.google.com/scholar?q=Nivre%2C+Joakim+and+Ryan+McDonald+%282008%29%2E+Integrating+Graph%2DBased+and+Transition%2DBased+Dependency+Parsers%2E+In+Proceedings+of+ACL%2D08%3A+HLT%2E+Columbus%2C+OH%2E)
21. Nivre, Joakim, Jens Nilsson and Johan Hall (2006). Talbanken05: A Swedish Treebank with Phrase Structure and Dependency Annotation. In Proceedings of LREC-06. Genoa, Italy. [find](https://scholar.google.com/scholar?q=Nivre%2C+Joakim%2C+Jens+Nilsson+and+Johan+Hall+%282006%29%2E+Talbanken05%3A+A+Swedish+Treebank+with+Phrase+Structure+and+Dependency+Annotation%2E+In+Proceedings+of+LREC%2D06%2E+Genoa%2C+Italy%2E)
22. Osborne, Miles (2002). Shallow Parsing using Noisy and Non-Stationary Training Material. In JMLR Special Issue on Machine Learning Approaches to Shallow Parsing, vol. 2, pp. 695– 719. [find](https://scholar.google.com/scholar?q=Osborne%2C+Miles+%282002%29%2E+Shallow+Parsing+using+Noisy+and+Non%2DStationary+Training+Material%2E+In+JMLR+Special+Issue+on+Machine+Learning+Approaches+to+Shallow+Parsing%2C+vol%2E+2%2C+pp%2E+695%E2%80%93+719%2E)
23. Padro, Lluis and Lluis Marquez (1998). On the Evaluation and Comparison of Taggers: the Effect of Noise in Testing Corpora. In Proceedings of ACL-COLING-98. San Francisco, CA. [find](https://scholar.google.com/scholar?q=Padro%2C+Lluis+and+Lluis+Marquez+%281998%29%2E+On+the+Evaluation+and+Comparison+of+Taggers%3A+the+Effect+of+Noise+in+Testing+Corpora%2E+In+Proceedings+of+ACL%2DCOLING%2D98%2E+San+Francisco%2C+CA%2E)
24. Taylor, Ann, Mitchell Marcus and Beatrice Santorini (2003). The Penn Treebank: An Overview. In Anne Abeillé (ed.), Treebanks: Building and using syntactically annotated corpora, Dordrecht: Kluwer, chap. 1, pp. 5–22. [find](https://scholar.google.com/scholar?q=Taylor%2C+Ann%2C+Mitchell+Marcus+and+Beatrice+Santorini+%282003%29%2E+The+Penn+Treebank%3A+An+Overview%2E+In+Anne+Abeille%CC%81+%28ed%2E%29%2C+Treebanks%3A+Building+and+using+syntactically+annotated+corpora%2C+Dordrecht%3A+Kluwer%2C+ch)
25. Torres Martins, André Filipe, Dipanjan Das, Noah A. Smith and Eric P. Xing (2008). Stacking Dependency Parsers. In Proceedings of EMNLP-08. Honolulu, Hawaii, pp. 157–166. [find](https://scholar.google.com/scholar?q=Torres+Martins%2C+Andre%CC%81+Filipe%2C+Dipanjan+Das%2C+Noah+A%2E+Smith+and+Eric+P%2E+Xing+%282008%29%2E+Stacking+Dependency+Parsers%2E+In+Proceedings+of+EMNLP%2D08%2E+Honolulu%2C+Hawaii%2C+pp%2E+157%E2%80%93166%2E)
26. Zavrel, Jakub, Walter Daelemans and Jorn Veensta (1997). Resolving PP attachment Ambiguities with Memory-Based Learning. In Proceedings of CoNLL-97. Madrid. [find](https://scholar.google.com/scholar?q=Zavrel%2C+Jakub%2C+Walter+Daelemans+and+Jorn+Veensta+%281997%29%2E+Resolving+PP+attachment+Ambiguities+with+Memory%2DBased+Learning%2E+In+Proceedings+of+CoNLL%2D97%2E+Madrid%2E)