[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/aclbulk-2001-gildea-corpus-variation-parser-performance/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / Corpus Variation and Parser Performance

# Corpus Variation and Parser Performance

**Authors:** [Daniel Gildea](https://md.mova.elamur.ai/free/en/docs/papers/authors/daniel-gildea/) · **Year:** 2001 · **Venue:** Proceedings of the 2001 Conference on Empirical Methods in Natural Language Processing **Link:** [aclanthology.org/W01-0521](https://aclanthology.org/W01-0521/) · **Licence:** [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/) — summary only

[**268** citations](https://openalex.org/works?filter=cites:W199541590) openalex · 2026-10-19[**138** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/aclbulk-2001-gildea-corpus-variation-parser-performance/) of the 140673 papers whose bibliographies we read[Proceedings of the 2001 Conference on Empirical Methods in Natural Language Processing](https://aclanthology.org/volumes/W01-05/)[licence](https://creativecommons.org/licenses/by-nc-sa/3.0/)

- [Publisher's page](https://aclanthology.org/W01-0521/)
- [PDF](https://aclanthology.org/W01-0521.pdf)
- [OpenAlex](https://openalex.org/W199541590)

**Authors (1)**

1. [Gildea, Daniel](https://openalex.org/A5020945377) [ORCID](https://orcid.org/0000-0002-7858-2624) · [Semantic Scholar](https://www.semanticscholar.org/author/1793218) · h 37 · 9308 citations · 190 works

Citations and author metrics from openalex (2026-10-19); authors with the same name are not always told apart.

## What it is about

The paper asks how much a statistical parser depends on the corpus it is trained on. It uses Collins' Model 1, a lexicalized generative parser, trained and tested on the Wall Street Journal (39,832 training sentences) and on the Brown corpus (about 21,800 training sentences, with every tenth sentence held out). A model trained on WSJ loses several points on Brown (recall 80.3 against 86.1 on WSJ), while a Brown-trained model does better on Brown than the WSJ model does. Removing the lexical-bigram parameters costs under 0.5 points on WSJ and leaves the cross-corpus results unchanged, so the bigrams are corpus-specific. Pruning the bigrams by a relative-entropy criterion removes 43% of the 736,000 parameters with a small loss.

## Ideas

- I1. Lexical-feature ablation across banks: train `en` on EWT only, then on EWT plus ESLSpok, and test on ATIS (63.0 now) and GUM (76.85, test only) with and without lexical-identity features in `en/src/parse.rs`; the paper predicts that lexical pairs add little across corpora.
- I2. Pruning curve as a size measure: prune lexical features by a count-times-divergence score to a fixed budget and report EWT test LAS against the number of kept features, for the shipped model size.
- I3. Corpus-specific feature listing: after training, list the ten most weighted lexical feature pairs per bank, sorted deterministically with `BTreeMap`, to see which features are domain-specific before the model is shipped.

## For Mova

The code is `en/src/parse.rs` (features) and `en/src/ptag.rs` (its lexicon). The chapters are [19. The lexicon: vocabulary, multiword expressions and register](https://md.mova.elamur.ai/free/en/docs/languages/en/19-lexicon-mwe-register/) and `37-register-and-variation.md`. Measure first: the LAS drop on ATIS and GUM when lexical features are removed, since the paper shows that matched data beats more unmatched data.

## Cite

The paper itself, not this card. BibTeX from the library's record of the work.

```
@inproceedings{gildea2001corpus,
  title = {{Corpus Variation and Parser Performance}},
  author = {Gildea, Daniel},
  booktitle = {Proceedings of the 2001 Conference on Empirical Methods in Natural Language Processing},
  year = {2001},
  url = {https://aclanthology.org/W01-0521/}
}
```

- [← previous An Algebra for Semantic Construction in Constraint-based Grammars](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2001-copestake-algebra-semantic-construction-constraint-based/)
- [next → Generation of VP Ellipsis: A Corpus-Based Approach](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2001-hardt-generation-vp-ellipsis-corpus-based/)
- [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/)

## References

18 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Douglas Biber. 1993. Using register-diversi ed corpora for general language studies. Computational Linguistics , 19(2):219{241, June. [find](https://scholar.google.com/scholar?q=Douglas+Biber%2E+1993%2E+Using+register%2Ddiversi+ed+corpora+for+general+language+studies%2E+Computational+Linguistics+%2C+19%282%29%3A219%7B241%2C+June%2E)
2. Eugene Charniak. 1997. Statistical parsing with a context-free grammar and word statistics. In AAAI97 , Brown University, Providence, Rhode Island, August. [find](https://scholar.google.com/scholar?q=Eugene+Charniak%2E+1997%2E+Statistical+parsing+with+a+context%2Dfree+grammar+and+word+statistics%2E+In+AAAI97+%2C+Brown+University%2C+Providence%2C+Rhode+Island%2C+August%2E)
3. Eugene Charniak. 2000. A maximum-entropy-inspired parser. In Proceedings of the 1st Annual Meeting of the North American Chapter of the ACL (NAACL) , Seattle, Washington. [find](https://scholar.google.com/scholar?q=Eugene+Charniak%2E+2000%2E+A+maximum%2Dentropy%2Dinspired+parser%2E+In+Proceedings+of+the+1st+Annual+Meeting+of+the+North+American+Chapter+of+the+ACL+%28NAACL%29+%2C+Seattle%2C+Washington%2E)
4. Michael Collins, Jan Hajic, Lance Ramshaw, and Christoph Tillmann. 1999. A statistical parser for czech. In Proceedings of the 37th Annual Meeting of the ACL , College Park, Maryland. [find](https://scholar.google.com/scholar?q=Michael+Collins%2C+Jan+Hajic%2C+Lance+Ramshaw%2C+and+Christoph+Tillmann%2E+1999%2E+A+statistical+parser+for+czech%2E+In+Proceedings+of+the+37th+Annual+Meeting+of+the+ACL+%2C+College+Park%2C+Maryland%2E)
5. Michael Collins. 1996. A new statistical parser based on bigram lexical dependencies. In Proceedings of the 34th Annual Meeting of the ACL . [find](https://scholar.google.com/scholar?q=Michael+Collins%2E+1996%2E+A+new+statistical+parser+based+on+bigram+lexical+dependencies%2E+In+Proceedings+of+the+34th+Annual+Meeting+of+the+ACL+%2E)
6. Michael Collins. 1997. Three generative, lexicalised models for statistical parsing. In Proceedings of the 35th Annual Meeting of the ACL . [find](https://scholar.google.com/scholar?q=Michael+Collins%2E+1997%2E+Three+generative%2C+lexicalised+models+for+statistical+parsing%2E+In+Proceedings+of+the+35th+Annual+Meeting+of+the+ACL+%2E)
7. Michael Collins. 1999. Head-Driven Statistical Models for Natural Language Parsing . Ph.D. thesis, University of Pennsylvania, Philadelphia. [find](https://scholar.google.com/scholar?q=Michael+Collins%2E+1999%2E+Head%2DDriven+Statistical+Models+for+Natural+Language+Parsing+%2E+Ph%2ED%2E+thesis%2C+University+of+Pennsylvania%2C+Philadelphia%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/cl-2003-collins-head-driven-statistical-models-natural/)
8. Michael Collins. 2000. Discriminative reranking for natural language parsing. In Proceedings of the ICML . [find](https://scholar.google.com/scholar?q=Michael+Collins%2E+2000%2E+Discriminative+reranking+for+natural+language+parsing%2E+In+Proceedings+of+the+ICML+%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/collins-2000-discriminative-reranking-natural-language-parsing/)
9. Rebecca Hwa. 1999. Supervised grammar induction using training data with limited constituent information. In Proceedings of the 37th Annual Meeting of the ACL , College Park, Maryland. [find](https://scholar.google.com/scholar?q=Rebecca+Hwa%2E+1999%2E+Supervised+grammar+induction+using+training+data+with+limited+constituent+information%2E+In+Proceedings+of+the+37th+Annual+Meeting+of+the+ACL+%2C+College+Park%2C+Maryland%2E)
10. David Magerman. 1995. Statistical decision-tree models for parsing. In Proceedings of the 33rd Annual Meeting of the ACL . [find](https://scholar.google.com/scholar?q=David+Magerman%2E+1995%2E+Statistical+decision%2Dtree+models+for+parsing%2E+In+Proceedings+of+the+33rd+Annual+Meeting+of+the+ACL+%2E)
11. Ivan A. Melcuk. 1988. Dependency Syntax: Theory and Practice . State University of New York Press. [find](https://scholar.google.com/scholar?q=Ivan+A%2E+Me%13l%15cuk%2E+1988%2E+Dependency+Syntax%3A+Theory+and+Practice+%2E+State+University+of+New+York+Press%2E)
12. Adwait Ratnaparkhi. 1997. A linear observed time statistical parser based on maximum entropy models. In Proceedings of the Second Conference on Empirical Methods in Natural Language Processing . [find](https://scholar.google.com/scholar?q=Adwait+Ratnaparkhi%2E+1997%2E+A+linear+observed+time+statistical+parser+based+on+maximum+entropy+models%2E+In+Proceedings+of+the+Second+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%2E)
13. Douglas Roland and Daniel Jurafsky. 1998. How verb subcategorization frequencies are a ected by corpus choice. In Proceedings of COLING/ACL , pages 1122{1128. [find](https://scholar.google.com/scholar?q=Douglas+Roland+and+Daniel+Jurafsky%2E+1998%2E+How+verb+subcategorization+frequencies+are+a+ected+by+corpus+choice%2E+In+Proceedings+of+COLING%2FACL+%2C+pages+1122%7B1128%2E)
14. Douglas Roland, Daniel Jurafsky, Lise Menn, Susanne Gahl, Elizabeth Elder, and Chris Riddoch. 2000. Verb subcategorization frequency di erences between business-news and balanced corpora: the role of verb sense. In Proceedings of the Association for Computational Linguistics (ACL-2000) Workshop on Comparing Corpora . [find](https://scholar.google.com/scholar?q=Douglas+Roland%2C+Daniel+Jurafsky%2C+Lise+Menn%2C+Susanne+Gahl%2C+Elizabeth+Elder%2C+and+Chris+Riddoch%2E+2000%2E+Verb+subcategorization+frequency+di+erences+between+business%2Dnews+and+balanced+corpora%3A+the+role+of+)
15. Kristie Seymore and Roni Rosenfeld. 1996. Scalable backo language models. In ICSLP-96 , volume 1, pages 232{235, Philadelphia. [find](https://scholar.google.com/scholar?q=Kristie+Seymore+and+Roni+Rosenfeld%2E+1996%2E+Scalable+backo+language+models%2E+In+ICSLP%2D96+%2C+volume+1%2C+pages+232%7B235%2C+Philadelphia%2E)
16. Daniel Sleator and Davy Temperley. 1993. Parsing english with a link grammar. In Third International Workshop on Parsing Technologies , August. [find](https://scholar.google.com/scholar?q=Daniel+Sleator+and+Davy+Temperley%2E+1993%2E+Parsing+english+with+a+link+grammar%2E+In+Third+International+Workshop+on+Parsing+Technologies+%2C+August%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/parsers-1993-sleator-parsing-english-link-grammar/)
17. A. Stolcke, C. Chelba, D. Engle, V. Jimenez, L. Mangu, H. Printz, E. Ristad, R. Rosenfeld, D. Wu, F. Jelinek, and S. Khudanpur. 1996. Dependency language modeling. Summer Workshop Final Report 24, Center for Language and Speech Processing, Johns Hopkins University, Baltimore, April. [find](https://scholar.google.com/scholar?q=A%2E+Stolcke%2C+C%2E+Chelba%2C+D%2E+Engle%2C+V%2E+Jimenez%2C+L%2E+Mangu%2C+H%2E+Printz%2C+E%2E+Ristad%2C+R%2E+Rosenfeld%2C+D%2E+Wu%2C+F%2E+Jelinek%2C+and+S%2E+Khudanpur%2E+1996%2E+Dependency+language+modeling%2E+Summer+Workshop+Final+Report+24%2C+Cen)
18. Andreas Stolcke. 1998. Entropy-based pruning of backo language models. In Proc. DARPA Broadcast News Transcription and Understanding Workshop , pages 270{274, Lansdowne, Va. [find](https://scholar.google.com/scholar?q=Andreas+Stolcke%2E+1998%2E+Entropy%2Dbased+pruning+of+backo+language+models%2E+In+Proc%2E+DARPA+Broadcast+News+Transcription+and+Understanding+Workshop+%2C+pages+270%7B274%2C+Lansdowne%2C+Va%2E)