[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/lingbulk-2016-daiber-denoised-web-treebank-evaluating-dependency/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / The Denoised Web Treebank: Evaluating Dependency Parsing under Noisy Input Conditions

# The Denoised Web Treebank: Evaluating Dependency Parsing under Noisy Input Conditions

**Authors:** Joachim Daiber, [Rob van der Goot](https://md.mova.elamur.ai/free/en/docs/papers/authors/rob-van-der-goot/) · **Year:** 2016 · **Venue:** Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16) **Link:** [aclanthology.org/L16-1102](https://aclanthology.org/L16-1102/) · **Licence:** [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) — summary only

[**14** citations](https://openalex.org/works?filter=cites:W7160245132) openalex · 2026-10-19[**5** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/lingbulk-2016-daiber-denoised-web-treebank-evaluating-dependency/) of the 140673 papers whose bibliographies we read[Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)](https://aclanthology.org/volumes/L16-1/)[licence](https://creativecommons.org/licenses/by-nc/4.0/)

- [Publisher's page](https://aclanthology.org/L16-1102/)
- [PDF](https://aclanthology.org/L16-1102.pdf)
- [DOI 10.63317/2w9ohr48zi4a](https://doi.org/10.63317/2w9ohr48zi4a)
- [OpenAlex](https://openalex.org/W7160245132)

**Authors (2)**

1. [Joachim Daiber](https://openalex.org/A5076793943) [ORCID](https://orcid.org/0000-0003-2151-6826) · [Semantic Scholar](https://www.semanticscholar.org/author/1696803) · h 6 · 687 citations · 18 works
2. [Rob van der Goot](https://openalex.org/A5039811894) [ORCID](https://orcid.org/0009-0003-1999-4156) · [Semantic Scholar](https://www.semanticscholar.org/author/3449407) · h 17 · 870 citations · 98 works

Citations and author metrics from openalex (2026-10-19); authors with the same name are not always told apart.

## What it is about

The authors build a 500-tweet English treebank (250 development, 250 test) with a normalization layer: each tweet keeps its original tokens, and a normalized sentence with word alignments is parsed. Insertions, deletions and zero copulas are allowed in the normalized side, so standard UAS and LAS no longer apply; the paper defines an aligned precision, recall and F1 that reduce to UAS and LAS when tokens match one to one. On the development set, a graph-based MST parser with coarse Twitter tags plus a fine tagger reaches 72.41 unlabeled F1 without normalization, 76.17 with Twitter-specific syntax rules, and 77.08 with lexical normalization combined with machine translation. Gold normalization and gold tags give 79.28, so the normalization gain is close to the gold upper bound.

## Ideas

- I1. A deterministic normalization layer in Rust before `en/src/tok.rs` (lexical rules and a short-form list, in the spirit of the paper's unsupervised lexical model); measure the aligned F1 on the Denoised Web Treebank development set, test only, against the unnormalized `en` parse.
- I2. `en/src/ud.rs` and `tools/treeagree`: Twitter-specific rules (leading usernames, RT markers, hashtags and final URLs attached to the main verb as an unclassified relation); measure how many `dep` labels and gate violations appear on web text before and after the rules.
- I3. Aligned evaluation in `en parse-eval`: report the aligned F1 when the tokens of a system differ from gold, so that normalization experiments are scored on the original text; measure the score agreement with UAS and LAS on EWT, where they must match.
- I4. `en/src/ptag.rs`: combine a fine tagger with a coarse, domain-specific one by n-best voting, as the paper does; measure UPOS on web text (test only) with the vote and without it.

## For Mova

Touches `languages/en/19-lexicon-mwe-register`, `37-register-and-variation` and `38-dialogue-and-discourse-markers`, and the `en` crate (`tok.rs`, `ud.rs`, `ptag.rs`) and `tools/treeagree`. The treebank is NC: use it for tests and measurements only, never in a shipped model. Measure first the unnormalized `en` aligned F1 on the test half, run twice to confirm determinism.

Topics: [Dependency parsing](https://md.mova.elamur.ai/free/en/docs/papers/topics/dependency-parsing/) · [Treebanks](https://md.mova.elamur.ai/free/en/docs/papers/topics/treebanks/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.63317/2w9ohr48zi4a](https://doi.org/10.63317/2w9ohr48zi4a)

```
@inproceedings{daiber2016denoised,
  title = {{The Denoised Web Treebank: Evaluating Dependency Parsing under Noisy Input Conditions}},
  author = {Daiber, Joachim and Goot, Rob van der},
  booktitle = {Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)},
  year = {2016},
  doi = {10.63317/2w9ohr48zi4a},
  url = {https://doi.org/10.63317/2w9ohr48zi4a}
}
```

- [← previous Prepositional Phrase Attachment Problem Revisited: How VERBNET Can Help](https://md.mova.elamur.ai/free/en/docs/papers/lingbulk-2015-bailey-prepositional-phrase-attachment-problem-revisited/)
- [next → Prepositional Phrase Attachment over Word Embedding Products](https://md.mova.elamur.ai/free/en/docs/papers/lingbulk-2017-madhyastha-prepositional-phrase-attachment-over-word/)
- [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/)

## References

24 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aw, A., Zhang, M., Xiao, J., and Su, J. (2006). A phrase-based statistical model for SMS text normalization. In Proceedings of the COLING/ACL on Main conference poster sessions, pages 33–40. [find](https://scholar.google.com/scholar?q=Aw%2C+A%2E%2C+Zhang%2C+M%2E%2C+Xiao%2C+J%2E%2C+and+Su%2C+J%2E+%282006%29%2E+A+phrase%2Dbased+statistical+model+for+SMS+text+normalization%2E+In+Proceedings+of+the+COLING%2FACL+on+Main+conference+poster+sessions%2C+pages+33%E2%80%9340%2E)
2. Bies, A., Ferguson, M., Katz, K., MacIntyre, R., Tredinnick, V., Kim, G., Marcinkiewicz, M. A., and Schasberger, B. (1995). Bracketing guidelines for Treebank II style Penn Treebank project. Technical report, University of Pennsylvania. [find](https://scholar.google.com/scholar?q=Bies%2C+A%2E%2C+Ferguson%2C+M%2E%2C+Katz%2C+K%2E%2C+MacIntyre%2C+R%2E%2C+Tredinnick%2C+V%2E%2C+Kim%2C+G%2E%2C+Marcinkiewicz%2C+M%2E+A%2E%2C+and+Schasberger%2C+B%2E+%281995%29%2E+Bracketing+guidelines+for+Treebank+II+style+Penn+Treebank+project%2E+Technical)
3. Bies, A., Mott, J., Warner, C., and Kulick, S. (2012). Bracketing webtext: An addendum to Penn Treebank II guidelines. Technical report, Linguistic Data Consortium. [find](https://scholar.google.com/scholar?q=Bies%2C+A%2E%2C+Mott%2C+J%2E%2C+Warner%2C+C%2E%2C+and+Kulick%2C+S%2E+%282012%29%2E+Bracketing+webtext%3A+An+addendum+to+Penn+Treebank+II+guidelines%2E+Technical+report%2C+Linguistic+Data+Consortium%2E)
4. Brants, T. (2000). TnT: a statistical part-of-speech tagger. In Proceedings of the Sixth Conference on Applied Natural Language Processing, ANLC ’00, pages 224–231, Stroudsburg, PA, USA. [find](https://scholar.google.com/scholar?q=Brants%2C+T%2E+%282000%29%2E+TnT%3A+a+statistical+part%2Dof%2Dspeech+tagger%2E+In+Proceedings+of+the+Sixth+Conference+on+Applied+Natural+Language+Processing%2C+ANLC+%E2%80%9900%2C+pages+224%E2%80%93231%2C+Stroudsburg%2C+PA%2C+USA%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/brants-2000-tnt-statistical-part-speech-tagger/)
5. Buchholz, S. and Marsi, E. (2006). CoNLL-X shared task on multilingual dependency parsing. In Proceedings of the Tenth Conference on Computational Natural Language Learning, pages 149–164. [find](https://scholar.google.com/scholar?q=Buchholz%2C+S%2E+and+Marsi%2C+E%2E+%282006%29%2E+CoNLL%2DX+shared+task+on+multilingual+dependency+parsing%2E+In+Proceedings+of+the+Tenth+Conference+on+Computational+Natural+Language+Learning%2C+pages+149%E2%80%93164%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/parsers-2006-buchholz-conll-x-shared-task-multilingual/)
6. De Marneffe, M.-C., MacCartney, B., Manning, C. D., et al. (2006). Generating typed dependency parses from phrase structure parses. In Proceedings of LREC, volume 6, pages 449–454. [find](https://scholar.google.com/scholar?q=De+Marneffe%2C+M%2E%2DC%2E%2C+MacCartney%2C+B%2E%2C+Manning%2C+C%2E+D%2E%2C+et+al%2E+%282006%29%2E+Generating+typed+dependency+parses+from+phrase+structure+parses%2E+In+Proceedings+of+LREC%2C+volume+6%2C+pages+449%E2%80%93454%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/map-2006-marneffe-generating-typed-dependency-parses-phrase/)
7. Efron, B. and Tibshirani, R. (1993). An Introduction to the Bootstrap. Chapman & Hall, New York. [find](https://scholar.google.com/scholar?q=Efron%2C+B%2E+and+Tibshirani%2C+R%2E+%281993%29%2E+An+Introduction+to+the+Bootstrap%2E+Chapman+%26+Hall%2C+New+York%2E)
8. Federico, M. and Cettolo, M. (2007). Efficient handling of n-gram language models for statistical machine translation. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 88–95. [find](https://scholar.google.com/scholar?q=Federico%2C+M%2E+and+Cettolo%2C+M%2E+%282007%29%2E+Efficient+handling+of+n%2Dgram+language+models+for+statistical+machine+translation%2E+In+Proceedings+of+the+Second+Workshop+on+Statistical+Machine+Translation%2C+pages+8)
9. Foster, J., Çetinoglu, Ö., Wagner, J., Le Roux, J., Nivre, J., Hogan, D., and van Genabith, J. (2011a). From news to comment: Resources and benchmarks for parsing the language of web 2.0. In Proceedings of 5th International Joint Conference on Natural Language Processing, pages 893–901, Chiang Mai, Thailand, November. Asian Federation of Natural Language Processing. [find](https://scholar.google.com/scholar?q=Foster%2C+J%2E%2C+C%CC%A7etinoglu%2C+O%CC%88%2E%2C+Wagner%2C+J%2E%2C+Le+Roux%2C+J%2E%2C+Nivre%2C+J%2E%2C+Hogan%2C+D%2E%2C+and+van+Genabith%2C+J%2E+%282011a%29%2E+From+news+to+comment%3A+Resources+and+benchmarks+for+parsing+the+language+of+web+2%2E0%2E+In+Proceed)
10. Foster, J., Çetinoglu, Ö., Wagner, J., Le Roux, J., Hogan, S., Nivre, J., Hogan, D., and Van Genabith, J. (2011b). #hardtoparse: POS Tagging and Parsing the Twitterverse. In AAAI 2011 Workshop On Analyzing Microtext, pages 20–25, United States. [find](https://scholar.google.com/scholar?q=Foster%2C+J%2E%2C+C%CC%A7etinoglu%2C+O%CC%88%2E%2C+Wagner%2C+J%2E%2C+Le+Roux%2C+J%2E%2C+Hogan%2C+S%2E%2C+Nivre%2C+J%2E%2C+Hogan%2C+D%2E%2C+and+Van+Genabith%2C+J%2E+%282011b%29%2E+%23hardtoparse%3A+POS+Tagging+and+Parsing+the+Twitterverse%2E+In+AAAI+2011+Workshop+On+An)
11. Gimpel, K., Schneider, N., O’Connor, B., Das, D., Mills, D., Eisenstein, J., Heilman, M., Yogatama, D., Flanigan, J., and Smith, N. A. (2011). Part-of-speech tagging for Twitter: Annotation, features, and experiments. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 42–47. [find](https://scholar.google.com/scholar?q=Gimpel%2C+K%2E%2C+Schneider%2C+N%2E%2C+O%E2%80%99Connor%2C+B%2E%2C+Das%2C+D%2E%2C+Mills%2C+D%2E%2C+Eisenstein%2C+J%2E%2C+Heilman%2C+M%2E%2C+Yogatama%2C+D%2E%2C+Flanigan%2C+J%2E%2C+and+Smith%2C+N%2E+A%2E+%282011%29%2E+Part%2Dof%2Dspeech+tagging+for+Twitter%3A+Annotation%2C+features%2C)
12. Han, B. and Baldwin, T. (2011). Lexical normalisation of short text messages: Makn sens a #twitter. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 368–378. [find](https://scholar.google.com/scholar?q=Han%2C+B%2E+and+Baldwin%2C+T%2E+%282011%29%2E+Lexical+normalisation+of+short+text+messages%3A+Makn+sens+a+%23twitter%2E+In+Proceedings+of+the+49th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+36)
13. Jurafsky, D. and Martin, J. H. (2000). Speech and Language Processing An Introduction to Natural Language Processing, Computational Linguistics, and Speech. Pearson Education. [find](https://scholar.google.com/scholar?q=Jurafsky%2C+D%2E+and+Martin%2C+J%2E+H%2E+%282000%29%2E+Speech+and+Language+Processing+An+Introduction+to+Natural+Language+Processing%2C+Computational+Linguistics%2C+and+Speech%2E+Pearson+Education%2E)
14. Kaljahi, R., Foster, J., Roturier, J., Ribeyre, C., Lynn, T., and Le Roux, J. (2015). Foreebank: Syntactic analysis of customer support forums. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1341–1347, Lisbon, Portugal, September. [find](https://scholar.google.com/scholar?q=Kaljahi%2C+R%2E%2C+Foster%2C+J%2E%2C+Roturier%2C+J%2E%2C+Ribeyre%2C+C%2E%2C+Lynn%2C+T%2E%2C+and+Le+Roux%2C+J%2E+%282015%29%2E+Foreebank%3A+Syntactic+analysis+of+customer+support+forums%2E+In+Proceedings+of+the+2015+Conference+on+Empirical+Metho)
15. Khan, M., Dickinson, M., and Kübler, S. (2013). Towards domain adaptation for parsing web data. In Proceedings of the International Conference Recent Advances in Natural Language Processing RANLP 2013, pages 357–364, Hissar, Bulgaria, September. INCOMA Ltd. Shoumen, Bulgaria. [find](https://scholar.google.com/scholar?q=Khan%2C+M%2E%2C+Dickinson%2C+M%2E%2C+and+Ku%CC%88bler%2C+S%2E+%282013%29%2E+Towards+domain+adaptation+for+parsing+web+data%2E+In+Proceedings+of+the+International+Conference+Recent+Advances+in+Natural+Language+Processing+RANLP+201)
16. Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., et al. (2007). Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions, pages 177–180. [find](https://scholar.google.com/scholar?q=Koehn%2C+P%2E%2C+Hoang%2C+H%2E%2C+Birch%2C+A%2E%2C+Callison%2DBurch%2C+C%2E%2C+Federico%2C+M%2E%2C+Bertoldi%2C+N%2E%2C+Cowan%2C+B%2E%2C+Shen%2C+W%2E%2C+Moran%2C+C%2E%2C+Zens%2C+R%2E%2C+et+al%2E+%282007%29%2E+Moses%3A+Open+source+toolkit+for+statistical+machine+translation)
17. Kong, L., Schneider, N., Swayamdipta, S., Bhatia, A., Dyer, C., and Smith, N. A. (2014). A dependency parser for Tweets. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1001–1012, Doha, Qatar, October. [find](https://scholar.google.com/scholar?q=Kong%2C+L%2E%2C+Schneider%2C+N%2E%2C+Swayamdipta%2C+S%2E%2C+Bhatia%2C+A%2E%2C+Dyer%2C+C%2E%2C+and+Smith%2C+N%2E+A%2E+%282014%29%2E+A+dependency+parser+for+Tweets%2E+In+Proceedings+of+the+2014+Conference+on+Empirical+Methods+in+Natural+Language+) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2014-kong-dependency-parser-tweets/)
18. Liu, X., Zhou, M., Wei, F., Fu, Z., and Zhou, X. (2012). Joint inference of named entity recognition and normalization for tweets. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pages 526–535. [find](https://scholar.google.com/scholar?q=Liu%2C+X%2E%2C+Zhou%2C+M%2E%2C+Wei%2C+F%2E%2C+Fu%2C+Z%2E%2C+and+Zhou%2C+X%2E+%282012%29%2E+Joint+inference+of+named+entity+recognition+and+normalization+for+tweets%2E+In+Proceedings+of+the+50th+Annual+Meeting+of+the+Association+for+Comp)
19. McDonald, R. T., Pereira, F., Ribarov, K., and Hajic, J. (2005). Non-projective dependency parsing using spanning tree algorithms. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=McDonald%2C+R%2E+T%2E%2C+Pereira%2C+F%2E%2C+Ribarov%2C+K%2E%2C+and+Hajic%2C+J%2E+%282005%29%2E+Non%2Dprojective+dependency+parsing+using+spanning+tree+algorithms%2E+In+Proceedings+of+the+Conference+on+Human+Language+Technology+and+Emp) [our card](https://md.mova.elamur.ai/free/en/docs/papers/narrative-2005-mcdonald-non-projective-dependency-parsing-spanning/)
20. Och, F. J. (2003). Minimum error rate training in statistical machine translation. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, volume 1, pages 160–167. [find](https://scholar.google.com/scholar?q=Och%2C+F%2E+J%2E+%282003%29%2E+Minimum+error+rate+training+in+statistical+machine+translation%2E+In+Proceedings+of+the+41st+Annual+Meeting+on+Association+for+Computational+Linguistics%2C+volume+1%2C+pages+160%E2%80%93167%2E)
21. Petrov, S. and McDonald, R. (2012). Overview of the 2012 shared task on parsing the web. In Notes of the First Workshop on Syntactic Analysis of Non-Canonical Language (SANCL), volume 59. [find](https://scholar.google.com/scholar?q=Petrov%2C+S%2E+and+McDonald%2C+R%2E+%282012%29%2E+Overview+of+the+2012+shared+task+on+parsing+the+web%2E+In+Notes+of+the+First+Workshop+on+Syntactic+Analysis+of+Non%2DCanonical+Language+%28SANCL%29%2C+volume+59%2E)
22. Prins, R. P. (2005). Finite-state pre-processing for natural language analysis. Ph.D. thesis, University of Groningen. [find](https://scholar.google.com/scholar?q=Prins%2C+R%2E+P%2E+%282005%29%2E+Finite%2Dstate+pre%2Dprocessing+for+natural+language+analysis%2E+Ph%2ED%2E+thesis%2C+University+of+Groningen%2E)
23. Raghunathan, K. and Krawczyk, S. (2009). Investigating SMS text normalization using statistical machine translation. Technical report. [find](https://scholar.google.com/scholar?q=Raghunathan%2C+K%2E+and+Krawczyk%2C+S%2E+%282009%29%2E+Investigating+SMS+text+normalization+using+statistical+machine+translation%2E+Technical+report%2E)
24. Silveira, N., Dozat, T., de Marneffe, M.-C., Bowman, S., Connor, M., Bauer, J., and Manning, C. (2014). A gold standard dependency corpus for English. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 2897–2904, Reykjavik, Iceland, May. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Silveira%2C+N%2E%2C+Dozat%2C+T%2E%2C+de+Marneffe%2C+M%2E%2DC%2E%2C+Bowman%2C+S%2E%2C+Connor%2C+M%2E%2C+Bauer%2C+J%2E%2C+and+Manning%2C+C%2E+%282014%29%2E+A+gold+standard+dependency+corpus+for+English%2E+In+Proceedings+of+the+Ninth+International+Confere)