[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/enud-2011-manning-part-speech-tagging-97-100/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / Part-of-Speech Tagging from 97% to 100%: Is It Time for Some Linguistics?

# Part-of-Speech Tagging from 97% to 100%: Is It Time for Some Linguistics?

**Authors:** [Christopher D. Manning](https://md.mova.elamur.ai/free/en/docs/papers/authors/christopher-d-manning/) · **Year:** 2011 · **Venue:** Computational Linguistics and Intelligent Text Processing, 171-189 **Link:** [nlp.stanford.edu/pubs/CICLing2011-manning-tagging.pdf](https://nlp.stanford.edu/pubs/CICLing2011-manning-tagging.pdf) · **Licence:** copyrighted — summary only

[**408** citations](https://openalex.org/works?filter=cites:W1515847863) openalex-snapshot · 2026-10-14[**102** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/enud-2011-manning-part-speech-tagging-97-100/) of the 140673 papers whose bibliographies we read[Computational Linguistics and Intelligent Text Processing, 171-189](https://openalex.org/S106296714)

- [Publisher's page](https://doi.org/10.1007/978-3-642-19400-9_14)
- [PDF](https://nlp.stanford.edu/pubs/CICLing2011-manning-tagging.pdf)
- [DOI 10.1007/978-3-642-19400-9_14](https://doi.org/10.1007/978-3-642-19400-9_14)
- [OpenAlex](https://openalex.org/W1515847863)

**Authors (1)**

1. [Christopher D. Manning](https://openalex.org/A5046006076) [ORCID](https://orcid.org/0000-0001-6155-649X) · [Semantic Scholar](https://www.semanticscholar.org/author/144783904) · h 139 · 180101 citations · 770 works

Citations and author metrics from openalex-snapshot (2026-10-14); authors with the same name are not always told apart.

## What it is about

Taggers report about 97% token accuracy, but only about 56% of sentences are fully right, and a single error can break a downstream parser. Manning asks what it would take to reach close to 100%. He improves the Stanford tagger with richer features (word shapes and distributional similarity), reaching 97.28% token and 56.79% sentence accuracy in the best row of his table. Then he analyses the remaining errors by class. Inconsistent or missing standards account for 28%, difficult linguistics for 19.5%, cases a tagger could plausibly get right for 16%, gold-standard errors for 15.5%, underspecified categories for 12%, and lexicon gaps and unknown words for 4.5% each. His conclusion is that the largest remaining gains come from better linguistic taxonomies and consistent conventions, not from better learners, though he also notes limits to that route.

## Ideas

- I1. `en/src/ptag.rs` suspect labels: sort each tagger error on EWT dev into Manning's classes (inconsistent standard, gold wrong, difficult linguistics, lexicon gap, unknown word), and check that the suspect-label rule catches the inconsistent-standard class first.
- I2. `en tag-eval`: report sentence accuracy next to token accuracy (the paper's 56% against 97%) on EWT test and on dev; record the date with each number.
- I3. `en/src/morph.rs` and `en/src/tag.rs`: measure the share of tagger errors on words missing from the AGID and UD dictionary in code (lexicon gap and unknown word, about 9% in the paper), on EWT dev.
- I4. `en/src/ptag.rs`: test whether the tagger's errors on ESLSpok and CHILDES cluster in the inconsistent-standard class, counting them per class on each bank's dev part.

## For Mova

Touches [10. Word classes](https://md.mova.elamur.ai/free/en/docs/languages/en/10-word-classes/), [07. Punctuation, capital letters, abbreviations and numbers](https://md.mova.elamur.ai/free/en/docs/languages/en/07-punctuation-capitals-numbers/) and [19. The lexicon: vocabulary, multiword expressions and register](https://md.mova.elamur.ai/free/en/docs/languages/en/19-lexicon-mwe-register/). The paper is a strong argument for Mova's suspect-label approach: errors in the gold labels are themselves a class to measure. Start with I1 and I2 on dev, not on test, and report two runs.

Topics: [Part-of-speech tagging](https://md.mova.elamur.ai/free/en/docs/papers/topics/part-of-speech-tagging/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.1007/978-3-642-19400-9_14](https://doi.org/10.1007/978-3-642-19400-9_14)

```
@inproceedings{manning2011part,
  title = {{Part-of-Speech Tagging from 97\% to 100\%: Is It Time for Some Linguistics?}},
  author = {Manning, Christopher D.},
  booktitle = {Computational Linguistics and Intelligent Text Processing, 171-189},
  year = {2011},
  doi = {10.1007/978-3-642-19400-9\_14},
  url = {https://doi.org/10.1007/978-3-642-19400-9_14}
}
```

- [← previous Corpus Annotation for Parser Evaluation](https://md.mova.elamur.ai/free/en/docs/papers/enud-1999-carroll-corpus-annotation-parser-evaluation/)
- [next → An Empirical Comparison of Parsing Methods for Stanford Dependencies](https://md.mova.elamur.ai/free/en/docs/papers/enud-2014-kong-empirical-comparison-parsing-methods-stanford/)
- [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/)

## References

34 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Toutanova, K., Klein, D., Manning, C.D., Singer, Y.: Feature-rich part-of-speech tagging with a cyclic dependency network. In: NAACL 3. (2003) 252–259 [find](https://scholar.google.com/scholar?q=Toutanova%2C+K%2E%2C+Klein%2C+D%2E%2C+Manning%2C+C%2ED%2E%2C+Singer%2C+Y%2E%3A+Feature%2Drich+part%2Dof%2Dspeech+tagging+with+a+cyclic+dependency+network%2E+In%3A+NAACL+3%2E+%282003%29+252%E2%80%93259)
2. Shen, L., Satta, G., Joshi, A.: Guided learning for bidirectional sequence classification. In: ACL 2007. (2007) [find](https://scholar.google.com/scholar?q=Shen%2C+L%2E%2C+Satta%2C+G%2E%2C+Joshi%2C+A%2E%3A+Guided+learning+for+bidirectional+sequence+classification%2E+In%3A+ACL+2007%2E+%282007%29)
3. Spoustová, D.j., Hajič, J., Raab, J., Spousta, M.: Semi-supervised training for the averaged perceptron POS tagger. In: Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009). (2009) 763–771 [find](https://scholar.google.com/scholar?q=Spoustova%CC%81%2C+D%2Ej%2E%2C+Hajic%CC%8C%2C+J%2E%2C+Raab%2C+J%2E%2C+Spousta%2C+M%2E%3A+Semi%2Dsupervised+training+for+the+averaged+perceptron+POS+tagger%2E+In%3A+Proceedings+of+the+12th+Conference+of+the+European+Chapter+of+the+ACL+%28EACL+20)
4. Søgaard, A.: Simple semi-supervised training of part-of-speech taggers. In: Proceedings of the ACL 2010 Conference Short Papers. (2010) 205–208 [find](https://scholar.google.com/scholar?q=S%C3%B8gaard%2C+A%2E%3A+Simple+semi%2Dsupervised+training+of+part%2Dof%2Dspeech+taggers%2E+In%3A+Proceedings+of+the+ACL+2010+Conference+Short+Papers%2E+%282010%29+205%E2%80%93208)
5. Subramanya, A., Petrov, S., Pereira, F.: Efficient graph-based semi-supervised learning of structured tagging models. In: Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing. (2010) 167–176 [find](https://scholar.google.com/scholar?q=Subramanya%2C+A%2E%2C+Petrov%2C+S%2E%2C+Pereira%2C+F%2E%3A+Efficient+graph%2Dbased+semi%2Dsupervised+learning+of+structured+tagging+models%2E+In%3A+Proceedings+of+the+2010+Conference+on+Empirical+Methods+in+Natural+Language+Pr)
6. Collins, M.: Discriminative training methods for Hidden Markov Models: Theory and experiments with perceptron algorithms. In: EMNLP 2002. (2002) [find](https://scholar.google.com/scholar?q=Collins%2C+M%2E%3A+Discriminative+training+methods+for+Hidden+Markov+Models%3A+Theory+and+experiments+with+perceptron+algorithms%2E+In%3A+EMNLP+2002%2E+%282002%29) [our card](https://md.mova.elamur.ai/free/en/docs/papers/collins-2002-discriminative-training-methods-hidden-markov/)
7. Marcus, M.P., Santorini, B., Marcinkiewicz, M.A.: Building a large annotated corpus of English: The Penn treebank. Computational Linguistics 19 (1993) 313– 330 [find](https://scholar.google.com/scholar?q=Marcus%2C+M%2EP%2E%2C+Santorini%2C+B%2E%2C+Marcinkiewicz%2C+M%2EA%2E%3A+Building+a+large+annotated+corpus+of+English%3A+The+Penn+treebank%2E+Computational+Linguistics+19+%281993%29+313%E2%80%93+330) [our card](https://md.mova.elamur.ai/free/en/docs/papers/bank-1993-marcus-building-large-annotated-corpus-english/)
8. Finkel, J., Dingare, S., Manning, C., Nissim, M., Alex, B., Grover, C.: Exploring the boundaries: Gene and protein identification in biomedical text. BMC Bioinformatics 6 (Suppl 1) (2005) [find](https://scholar.google.com/scholar?q=Finkel%2C+J%2E%2C+Dingare%2C+S%2E%2C+Manning%2C+C%2E%2C+Nissim%2C+M%2E%2C+Alex%2C+B%2E%2C+Grover%2C+C%2E%3A+Exploring+the+boundaries%3A+Gene+and+protein+identification+in+biomedical+text%2E+BMC+Bioinformatics+6+%28Suppl+1%29+%282005%29)
9. Collins, M.: Ranking algorithms for named entity extraction: Boosting and the voted perceptron. In: ACL 40. (2002) 489–496 [find](https://scholar.google.com/scholar?q=Collins%2C+M%2E%3A+Ranking+algorithms+for+named+entity+extraction%3A+Boosting+and+the+voted+perceptron%2E+In%3A+ACL+40%2E+%282002%29+489%E2%80%93496)
10. Tsuruoka, Y., Tsujii, J.: Bidirectional inference with the easiest-first strategy for tagging sequence data. In: Proceedings of HLT/EMNLP 2005. (2005) 467–474 [find](https://scholar.google.com/scholar?q=Tsuruoka%2C+Y%2E%2C+Tsujii%2C+J%2E%3A+Bidirectional+inference+with+the+easiest%2Dfirst+strategy+for+tagging+sequence+data%2E+In%3A+Proceedings+of+HLT%2FEMNLP+2005%2E+%282005%29+467%E2%80%93474)
11. Clark, A.: Combining distributional and morphological information for part of speech induction. In: EACL 2003. (2003) 59–66 [find](https://scholar.google.com/scholar?q=Clark%2C+A%2E%3A+Combining+distributional+and+morphological+information+for+part+of+speech+induction%2E+In%3A+EACL+2003%2E+%282003%29+59%E2%80%9366)
12. Klein, D., Manning, C.D.: Accurate unlexicalized parsing. In: ACL 41. (2003) 423–430 [find](https://scholar.google.com/scholar?q=Klein%2C+D%2E%2C+Manning%2C+C%2ED%2E%3A+Accurate+unlexicalized+parsing%2E+In%3A+ACL+41%2E+%282003%29+423%E2%80%93430) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2003-klein-accurate-unlexicalized-parsing/)
13. MacKinlay, A.: The effects of part-of-speech tagsets on tagger performance. Honours thesis, Department of Computer Science and Software Engineering, University of Melbourne (2005) [find](https://scholar.google.com/scholar?q=MacKinlay%2C+A%2E%3A+The+effects+of+part%2Dof%2Dspeech+tagsets+on+tagger+performance%2E+Honours+thesis%2C+Department+of+Computer+Science+and+Software+Engineering%2C+University+of+Melbourne+%282005%29)
14. Church, K.W.: Current practice in part of speech tagging and suggestions for the future. In Mackie, A.W., McAuley, T.K., Simmons, C., eds.: For Henry Kučera: Studies in Slavic philology and computational linguistics. Number 6 in Papers in Slavic philology. Michigan Slavic Studies, Ann Arbor (1992) 13–48 [find](https://scholar.google.com/scholar?q=Church%2C+K%2EW%2E%3A+Current+practice+in+part+of+speech+tagging+and+suggestions+for+the+future%2E+In+Mackie%2C+A%2EW%2E%2C+McAuley%2C+T%2EK%2E%2C+Simmons%2C+C%2E%2C+eds%2E%3A+For+Henry+Kuc%CC%8Cera%3A+Studies+in+Slavic+philology+and+computati)
15. Magerman, D.M.: Natural language parsing as statistical pattern recognition. PhD thesis, Stanford University (1994) [find](https://scholar.google.com/scholar?q=Magerman%2C+D%2EM%2E%3A+Natural+language+parsing+as+statistical+pattern+recognition%2E+PhD+thesis%2C+Stanford+University+%281994%29)
16. Ratnaparkhi, A.: A maximum entropy model for part-of-speech tagging. In: EMNLP 1. (1996) 133–142 [find](https://scholar.google.com/scholar?q=Ratnaparkhi%2C+A%2E%3A+A+maximum+entropy+model+for+part%2Dof%2Dspeech+tagging%2E+In%3A+EMNLP+1%2E+%281996%29+133%E2%80%93142)
17. Abney, S., Schapire, R.E., Singer, Y.: Boosting applied to tagging and PP attachment. In Fung, P., Zhou, J., eds.: Proceedings of the 1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora. (1999) 38–45 [find](https://scholar.google.com/scholar?q=Abney%2C+S%2E%2C+Schapire%2C+R%2EE%2E%2C+Singer%2C+Y%2E%3A+Boosting+applied+to+tagging+and+PP+attachment%2E+In+Fung%2C+P%2E%2C+Zhou%2C+J%2E%2C+eds%2E%3A+Proceedings+of+the+1999+Joint+SIGDAT+Conference+on+Empirical+Methods+in+Natural+Langu)
18. Voutilainen, A., Järvinen, T.: Specifying a shallow grammatical representation for parsing purposes. In: 7th Conference of the European Chapter of the Association for Computational Linguistics. (1995) 210–214 [find](https://scholar.google.com/scholar?q=Voutilainen%2C+A%2E%2C+Ja%CC%88rvinen%2C+T%2E%3A+Specifying+a+shallow+grammatical+representation+for+parsing+purposes%2E+In%3A+7th+Conference+of+the+European+Chapter+of+the+Association+for+Computational+Linguistics%2E+%281995)
19. Samuelsson, C., Voutilainen, A.: Comparing a linguistic and a stochastic tagger. In: Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics. (1997) 246–253 [find](https://scholar.google.com/scholar?q=Samuelsson%2C+C%2E%2C+Voutilainen%2C+A%2E%3A+Comparing+a+linguistic+and+a+stochastic+tagger%2E+In%3A+Proceedings+of+the+35th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2E+%281997%29+246%E2%80%93253)
20. Dickinson, M., Meurers, W.D.: Detecting errors in part-of-speech annotation. In: Proceedings of the 10th Conference of the European Chapter of the Association for Computational Linguistics (EACL-03). (2003) [find](https://scholar.google.com/scholar?q=Dickinson%2C+M%2E%2C+Meurers%2C+W%2ED%2E%3A+Detecting+errors+in+part%2Dof%2Dspeech+annotation%2E+In%3A+Proceedings+of+the+10th+Conference+of+the+European+Chapter+of+the+Association+for+Computational+Linguistics+%28EACL%2D03%29%2E+)
21. Levy, R., Andrew, G.: Tregex and Tsurgeon: tools for querying and manipulating tree data structures. In: 5th International Conference on Language Resources and Evaluation (LREC 2006). (2006) [find](https://scholar.google.com/scholar?q=Levy%2C+R%2E%2C+Andrew%2C+G%2E%3A+Tregex+and+Tsurgeon%3A+tools+for+querying+and+manipulating+tree+data+structures%2E+In%3A+5th+International+Conference+on+Language+Resources+and+Evaluation+%28LREC+2006%29%2E+%282006%29)
22. Rohde, D.L.T.: Tgrep2 user manual. ms., MIT (2005) [find](https://scholar.google.com/scholar?q=Rohde%2C+D%2EL%2ET%2E%3A+Tgrep2+user+manual%2E+ms%2E%2C+MIT+%282005%29)
23. Santorini, B.: Part-of-speech tagging guidelines for the Penn treebank project. 3rd Revision, 2nd printing, Feb. 1995. University of Pennsylvania (1990) [find](https://scholar.google.com/scholar?q=Santorini%2C+B%2E%3A+Part%2Dof%2Dspeech+tagging+guidelines+for+the+Penn+treebank+project%2E+3rd+Revision%2C+2nd+printing%2C+Feb%2E+1995%2E+University+of+Pennsylvania+%281990%29) [our card](https://md.mova.elamur.ai/free/en/docs/papers/santorini-1990-part-of-speech-tagging-guidelines-penn-treebank/)
24. Moore, D.S.: Statistics: Concepts and Controversies. 3rd edn. W. H. Freeman, New York (1991) [find](https://scholar.google.com/scholar?q=Moore%2C+D%2ES%2E%3A+Statistics%3A+Concepts+and+Controversies%2E+3rd+edn%2E+W%2E+H%2E+Freeman%2C+New+York+%281991%29)
25. Ross, J.R.: A fake NP squish. In Bailey, C.J.N., Shuy, R.W., eds.: New Ways of Analyzing Variation in English. Georgetown University Press, Washington (1973) 96–140 [find](https://scholar.google.com/scholar?q=Ross%2C+J%2ER%2E%3A+A+fake+NP+squish%2E+In+Bailey%2C+C%2EJ%2EN%2E%2C+Shuy%2C+R%2EW%2E%2C+eds%2E%3A+New+Ways+of+Analyzing+Variation+in+English%2E+Georgetown+University+Press%2C+Washington+%281973%29+96%E2%80%93140)
26. Quirk, R., Greenbaum, S., Leech, G., Svartvik, J.: A Comprehensive Grammar of the English Language. Longman, London (1985) [find](https://scholar.google.com/scholar?q=Quirk%2C+R%2E%2C+Greenbaum%2C+S%2E%2C+Leech%2C+G%2E%2C+Svartvik%2C+J%2E%3A+A+Comprehensive+Grammar+of+the+English+Language%2E+Longman%2C+London+%281985%29)
27. Aarts, B.: Syntactic gradience: the nature of grammatical indeterminacy. Oxford University Press, Oxford (2007) [find](https://scholar.google.com/scholar?q=Aarts%2C+B%2E%3A+Syntactic+gradience%3A+the+nature+of+grammatical+indeterminacy%2E+Oxford+University+Press%2C+Oxford+%282007%29)
28. Abney, S.: Statistical methods and linguistics. In Klavans, J., Resnik, P., eds.: The Balancing Act. MIT Press, Cambridge, MA (1996) [find](https://scholar.google.com/scholar?q=Abney%2C+S%2E%3A+Statistical+methods+and+linguistics%2E+In+Klavans%2C+J%2E%2C+Resnik%2C+P%2E%2C+eds%2E%3A+The+Balancing+Act%2E+MIT+Press%2C+Cambridge%2C+MA+%281996%29)
29. Maling, J.: Transitive adjectives: A case of categorial reanalysis. In Heny, F., Richards, B., eds.: Linguistic Categories: Auxiliaries and Related Puzzles. Volume 1. D. Reidel, Dordrecht (1983) 253–289 [find](https://scholar.google.com/scholar?q=Maling%2C+J%2E%3A+Transitive+adjectives%3A+A+case+of+categorial+reanalysis%2E+In+Heny%2C+F%2E%2C+Richards%2C+B%2E%2C+eds%2E%3A+Linguistic+Categories%3A+Auxiliaries+and+Related+Puzzles%2E+Volume+1%2E+D%2E+Reidel%2C+Dordrecht+%281983%29+253%E2%80%932)
30. Harnad, S., ed.: Categorical perception : the groundwork of cognition. Cambridge University Press, Cambridge (1987) [find](https://scholar.google.com/scholar?q=Harnad%2C+S%2E%2C+ed%2E%3A+Categorical+perception+%3A+the+groundwork+of+cognition%2E+Cambridge+University+Press%2C+Cambridge+%281987%29)
31. Radford, A.: Transformational Grammar. Cambridge University Press, Cambridge (1988) [find](https://scholar.google.com/scholar?q=Radford%2C+A%2E%3A+Transformational+Grammar%2E+Cambridge+University+Press%2C+Cambridge+%281988%29)
32. Manning, C.D., Schütze, H.: Foundations of Statistical Natural Language Processing. MIT Press, Boston, MA (1999) [find](https://scholar.google.com/scholar?q=Manning%2C+C%2ED%2E%2C+Schu%CC%88tze%2C+H%2E%3A+Foundations+of+Statistical+Natural+Language+Processing%2E+MIT+Press%2C+Boston%2C+MA+%281999%29)
33. Bies, A., Ferguson, M., Katz, K., MacIntyre, R., and colleagues: Bracketing guidelines for Treebank II style: Penn treebank project. ms, University of Pennsylvania (1995) [find](https://scholar.google.com/scholar?q=Bies%2C+A%2E%2C+Ferguson%2C+M%2E%2C+Katz%2C+K%2E%2C+MacIntyre%2C+R%2E%2C+and+colleagues%3A+Bracketing+guidelines+for+Treebank+II+style%3A+Penn+treebank+project%2E+ms%2C+University+of+Pennsylvania+%281995%29)
34. Huddleston, R.D., Pullum, G.K.: The Cambridge Grammar of the English Language. Cambridge University Press, Cambridge (2002) [find](https://scholar.google.com/scholar?q=Huddleston%2C+R%2ED%2E%2C+Pullum%2C+G%2EK%2E%3A+The+Cambridge+Grammar+of+the+English+Language%2E+Cambridge+University+Press%2C+Cambridge+%282002%29) [our card](https://md.mova.elamur.ai/free/en/docs/papers/huddleston-2002-cambridge-grammar-english-language/)