[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/aclbulk-2009-honnibal-evaluating-statistical-ccg-parser-wikipedia/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / Evaluating a Statistical CCG Parser on Wikipedia

# Evaluating a Statistical CCG Parser on Wikipedia

**Authors:** [Matthew Honnibal](https://md.mova.elamur.ai/free/en/docs/papers/authors/matthew-honnibal/), Joel Nothman and [James R. Curran](https://md.mova.elamur.ai/free/en/docs/papers/authors/james-r-curran/) · **Year:** 2009 · **Venue:** Proceedings of the 2009 Workshop on The People's Web Meets NLP: Collaboratively Constructed Semantic Resources (People's Web) **Link:** [aclanthology.org/W09-3306](https://aclanthology.org/W09-3306/) · **Licence:** [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/) — summary only

[**14** citations](https://openalex.org/works?filter=cites:W2054867311) openalex-snapshot · 2026-10-14[**5** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/aclbulk-2009-honnibal-evaluating-statistical-ccg-parser-wikipedia/) of the 140673 papers whose bibliographies we read[Proceedings of the 2009 Workshop on The People's Web Meets NLP: Collaboratively Constructed Semantic Resources (People's Web)](https://aclanthology.org/volumes/W09-33/)[licence](https://creativecommons.org/licenses/by-nc-sa/3.0/)

- [Publisher's page](https://aclanthology.org/W09-3306/)
- [PDF](https://aclanthology.org/W09-3306.pdf)
- [DOI 10.3115/1699765.1699771](https://doi.org/10.3115/1699765.1699771)
- [OpenAlex](https://openalex.org/W2054867311)

**Authors (3)**

1. [Matthew Honnibal](https://openalex.org/A5081770499) [Semantic Scholar](https://www.semanticscholar.org/author/3182726) · h 10 · 1187 citations · 28 works
2. [Joel Nothman](https://openalex.org/A5079518059) [ORCID](https://orcid.org/0000-0003-1497-3842) · [Semantic Scholar](https://www.semanticscholar.org/author/3083916) · h 15 · 12369 citations · 54 works
3. James R. Curran [Semantic Scholar](https://www.semanticscholar.org/author/1733593)

Citations and author metrics from openalex-snapshot (2026-10-14); authors with the same name are not always told apart.

## What it is about

The authors ask how well a statistical CCG parser (C&C) works on Wikipedia text. They hand-label 200 random sentences, which took about 20 hours. Accuracy is 4.3% lower than on newswire, and the parser is 54% slower. Self-training on Simple English Wikipedia raises accuracy on Wikipedia by 0.8% and narrows the gap to 3.8%. Speed improves by 20% on newswire and by 34% on Wikipedia. At the measured speed a full Wikipedia dump would take about 52 days of CPU time. The main lesson is that domain adaptation for a lexicalised parser is mostly adaptation of the supertagger.

## Ideas

- I1. Domain drift probe: run `en parse-eval` on EWT test, on PUD, and on a 200-sentence sample of our own encyclopedic-style text, hand-labelled. Report the LAS drop, as the paper's 4.3% comparison does.
- I2. Self-training with silver trees: add `en` parses of plain school text to the perceptron, and measure LAS on EWT test with two identical runs. The paper's +0.8% is the reference size.
- I3. Speed table: report sentences per second for EWT and for the hand-labelled sample, and check whether any tag-set pruning gives a speed gain of the kind the paper reports.

## For Mova

The code is `en/src/parse.rs` and `en/src/ptag.rs`; use `en folds` for the decision. No Wikipedia text enters the build: the self-training source must be our own plain text. The paper's numbers are C&C numbers on CCG output and are not comparable with LAS.

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.3115/1699765.1699771](https://doi.org/10.3115/1699765.1699771)

```
@inproceedings{honnibal2009evaluating,
  title = {{Evaluating a Statistical CCG Parser on Wikipedia}},
  author = {Honnibal, Matthew and Nothman, Joel and Curran, James R.},
  booktitle = {Proceedings of the 2009 Workshop on The People's Web Meets NLP: Collaboratively Constructed Semantic Resources (People's Web)},
  year = {2009},
  doi = {10.3115/1699765.1699771},
  url = {https://doi.org/10.3115/1699765.1699771}
}
```

- [← previous Descriptive and Empirical Approaches to Capturing Underlying Dependencies among Parsing Errors](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2009-hara-descriptive-empirical-approaches-capturing-underlying/)
- [next → Automatic training of lemmatization rules that handle morphological changes in pre-, in- and suffixes alike](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2009-jongejan-automatic-training-lemmatization-rules-that/)
- [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/)

## References

16 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Srinivas Bangalore and Aravind Joshi. 1999. Supertagging: An approach to almost parsing. Computational Linguistics, 25(2):237–265. [find](https://scholar.google.com/scholar?q=Srinivas+Bangalore+and+Aravind+Joshi%2E+1999%2E+Supertagging%3A+An+approach+to+almost+parsing%2E+Computational+Linguistics%2C+25%282%29%3A237%E2%80%93265%2E)
2. Stephen Clark and James R. Curran. 2007. Wide-coverage efficient statistical parsing with CCG and log-linear models. Computational Linguistics, 33(4):493–552. [find](https://scholar.google.com/scholar?q=Stephen+Clark+and+James+R%2E+Curran%2E+2007%2E+Wide%2Dcoverage+efficient+statistical+parsing+with+CCG+and+log%2Dlinear+models%2E+Computational+Linguistics%2C+33%284%29%3A493%E2%80%93552%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/nlpgap-2007-clark-wide-coverage-efficient-statistical-parsing/)
3. Mark Dredze, John Blitzer, Partha Pratim Talukdar, Kuzman Ganchev, João Graca, and Fernando Pereira. 2007. Frustratingly hard domain adaptation for dependency parsing. In Proceedings of the CoNLL Shared Task Session of EMNLP-CoNLL 2007, pages 1051–1055. ACL, Prague, Czech Republic. [find](https://scholar.google.com/scholar?q=Mark+Dredze%2C+John+Blitzer%2C+Partha+Pratim+Talukdar%2C+Kuzman+Ganchev%2C+Joa%CC%83o+Graca%2C+and+Fernando+Pereira%2E+2007%2E+Frustratingly+hard+domain+adaptation+for+dependency+parsing%2E+In+Proceedings+of+the+CoNLL+Sha) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2007-dredze-frustratingly-hard-domain-adaptation-dependency/)
4. Jason Eisner. 1996. Efficient normal-form parsing for Combinatory Categorial Grammar. In Proceedings of the Association for Computational Linguistics, pages 79–86. Santa Cruz, CA, USA. [find](https://scholar.google.com/scholar?q=Jason+Eisner%2E+1996%2E+Efficient+normal%2Dform+parsing+for+Combinatory+Categorial+Grammar%2E+In+Proceedings+of+the+Association+for+Computational+Linguistics%2C+pages+79%E2%80%9386%2E+Santa+Cruz%2C+CA%2C+USA%2E)
5. Daniel Gildea. 2001. Corpus variation and parser performance. In Proceedings of the EMNLP Conference, pages 167–202. Pittsburgh, PA. [find](https://scholar.google.com/scholar?q=Daniel+Gildea%2E+2001%2E+Corpus+variation+and+parser+performance%2E+In+Proceedings+of+the+EMNLP+Conference%2C+pages+167%E2%80%93202%2E+Pittsburgh%2C+PA%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2001-gildea-corpus-variation-parser-performance/)
6. Julia Hockenmaier and Mark Steedman. 2005. CCGbank manual. Technical Report MS-CIS-05-09, Department of Computer Science, University of Pennsylvania. [find](https://scholar.google.com/scholar?q=Julia+Hockenmaier+and+Mark+Steedman%2E+2005%2E+CCGbank+manual%2E+Technical+Report+MS%2DCIS%2D05%2D09%2C+Department+of+Computer+Science%2C+University+of+Pennsylvania%2E)
7. Julia Hockenmaier and Mark Steedman. 2007. CCGbank: a corpus of CCG derivations and dependency structures extracted from the Penn Treebank. Computational Linguistics, 33(3):355–396. [find](https://scholar.google.com/scholar?q=Julia+Hockenmaier+and+Mark+Steedman%2E+2007%2E+CCGbank%3A+a+corpus+of+CCG+derivations+and+dependency+structures+extracted+from+the+Penn+Treebank%2E+Computational+Linguistics%2C+33%283%29%3A355%E2%80%93396%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/bank-2007-hockenmaier-ccgbank-corpus-ccg-derivations-dependency/)
8. Tibor Kiss and Jan Strunk. 2006. Unsupervised multilingual sentence boundary detection. Computational Linguistics, 32(4):485–525. [find](https://scholar.google.com/scholar?q=Tibor+Kiss+and+Jan+Strunk%2E+2006%2E+Unsupervised+multilingual+sentence+boundary+detection%2E+Computational+Linguistics%2C+32%284%29%3A485%E2%80%93525%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/cl-2006-kiss-unsupervised-multilingual-sentence-boundary-detection/)
9. Edward Loper and Steven Bird. 2002. NLTK: The natural language toolkit. [find](https://scholar.google.com/scholar?q=Edward+Loper+and+Steven+Bird%2E+2002%2E+NLTK%3A+The+natural+language+toolkit%2E)
10. Joakim Nivre, Johan Hall, Sandra Kübler, Ryan McDonald, Jens Nilsson, Sebastian Riedel, and Deniz Yuret. 2007. [find](https://scholar.google.com/scholar?q=Joakim+Nivre%2C+Johan+Hall%2C+Sandra+Ku%CC%88bler%2C+Ryan+McDonald%2C+Jens+Nilsson%2C+Sebastian+Riedel%2C+and+Deniz+Yuret%2E+2007%2E)
11. The CoNLL 2007 shared task on dependency parsing. In Proceedings of the CoNLL Shared Task Session, pages 915–932. Prague, Czech Republic. [find](https://scholar.google.com/scholar?q=The+CoNLL+2007+shared+task+on+dependency+parsing%2E+In+Proceedings+of+the+CoNLL+Shared+Task+Session%2C+pages+915%E2%80%93932%2E+Prague%2C+Czech+Republic%2E)
12. PediaPress. 2007. mwlib MediaWiki parsing library. [http://code.pediapress.com](http://code.pediapress.com). [↗](http://code.pediapress.com)
13. Laura Rimell and Stephen Clark. 2009. Porting a lexicalized-grammar parser to the biomedical domain. Journal of Biomedical Informatics. (in press). [find](https://scholar.google.com/scholar?q=Laura+Rimell+and+Stephen+Clark%2E+2009%2E+Porting+a+lexicalized%2Dgrammar+parser+to+the+biomedical+domain%2E+Journal+of+Biomedical+Informatics%2E+%28in+press%29%2E)
14. Mark Steedman. 2000. The Syntactic Process. The MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=Mark+Steedman%2E+2000%2E+The+Syntactic+Process%2E+The+MIT+Press%2C+Cambridge%2C+MA%2E)
15. Mark Steedman, Rebecca Hwa, Stephen Clark, Miles Osborne, Anoop Sarkar, Julia Hockenmaier, Paul Ruhlen, Steven Baker, and Jeremiah Crim. 2003. Example selection for bootstrapping statistical parsers. In Proceedings of HLT-NAACL 2003. Edmonton, Alberta. [find](https://scholar.google.com/scholar?q=Mark+Steedman%2C+Rebecca+Hwa%2C+Stephen+Clark%2C+Miles+Osborne%2C+Anoop+Sarkar%2C+Julia+Hockenmaier%2C+Paul+Ruhlen%2C+Steven+Baker%2C+and+Jeremiah+Crim%2E+2003%2E+Example+selection+for+bootstrapping+statistical+parsers%2E+)
16. Gisle Ytrestøl, Stephan Oepen, and Daniel Flickinger. 2009. Extracting and annotating Wikipedia sub-domains. In Proceedings of the 7th International Workshop on Treebanks and Linguistic Theories, pages 185–197. Groningen, Netherlands. [find](https://scholar.google.com/scholar?q=Gisle+Ytrest%C3%B8l%2C+Stephan+Oepen%2C+and+Daniel+Flickinger%2E+2009%2E+Extracting+and+annotating+Wikipedia+sub%2Ddomains%2E+In+Proceedings+of+the+7th+International+Workshop+on+Treebanks+and+Linguistic+Theories%2C+page)