[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/aclbulk-2009-toutanova-global-model-joint-lemmatization-part/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / A global model for joint lemmatization and part-of-speech prediction

# A global model for joint lemmatization and part-of-speech prediction

**Authors:** [Kristina Toutanova](https://md.mova.elamur.ai/free/en/docs/papers/authors/kristina-toutanova/), [Colin Cherry](https://md.mova.elamur.ai/free/en/docs/papers/authors/colin-cherry/) · **Year:** 2009 · **Venue:** Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP **Link:** [aclanthology.org/P09-1055](https://aclanthology.org/P09-1055/) · **Licence:** [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/) — summary only

[**41** citations](https://openalex.org/works?filter=cites:W2131514140) openalex · 2026-10-18[**17** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/aclbulk-2009-toutanova-global-model-joint-lemmatization-part/) of the 140673 papers whose bibliographies we read[Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP](https://aclanthology.org/volumes/P09-1/)[licence](https://creativecommons.org/licenses/by-nc-sa/3.0/)

- [Publisher's page](https://aclanthology.org/P09-1055/)
- [PDF](https://aclanthology.org/P09-1055.pdf)
- [DOI 10.3115/1687878.1687947](https://doi.org/10.3115/1687878.1687947)
- [OpenAlex](https://openalex.org/W2131514140)

**Authors (2)**

1. [Kristina Toutanova](https://openalex.org/A5053947885) [ORCID](https://orcid.org/0009-0005-3458-9049) · h 44 · 62936 citations · 154 works
2. [Colin Cherry](https://openalex.org/A5048302740) h 44 · 8948 citations · 208 works

Citations and author metrics from openalex (2026-10-18); authors with the same name are not always told apart.

## What it is about

The authors predict lemmas and part-of-speech tags together, because the two depend on each other. Their model combines a tag-set model and a lemmatization model, and adds joint features over a dependency structure of the words, so the decision for one word can use its neighbours. It uses only morphological lexicons and unlabeled data, so the tagger is partly supervised. The joint model is tested on English, Bulgarian, Slovene and Czech. It reduces lemmatization error by up to 19%, tag error by up to 26%, and the error on the full morphological analysis by up to 22.6%, compared with the separate models.

## Ideas

- I1. `en/src/morph.rs` with `en/src/ptag.rs`: for each form with several lemma candidates (for example "found" as verb or noun), score the candidate lemma with the tag decided by the tagger, and keep the best pair. Measure lemma accuracy on EWT test with and without the tag.
- I2. Use the dependency structure already built by `en/src/parse.rs`: the lemma of a word gets a feature from its head's tag. Measure the lemma error reduction on EWT test, and the change in tag accuracy, so the two gains are reported separately.
- I3. Check the paper's claim of joint gain on English alone: count the lemma and tag errors of the separate models on EWT test, and report the number of errors that a joint score would fix before building it.

## For Mova

The code is `en/src/morph.rs`, `en/src/ptag.rs` and `en/src/parse.rs`, with `en morph-eval`, `en tag-eval` and `en parse-eval`. The chapter is [10. Word classes](https://md.mova.elamur.ai/free/en/docs/languages/en/10-word-classes/). The paper's models use lexicons and a log-linear model, which fit Mova's rules. Measure first on EWT test: lemma and tag accuracy with the joint score, twice.

Topics: [Part-of-speech tagging](https://md.mova.elamur.ai/free/en/docs/papers/topics/part-of-speech-tagging/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.3115/1687878.1687947](https://doi.org/10.3115/1687878.1687947)

```
@inproceedings{toutanova2009global,
  title = {{A global model for joint lemmatization and part-of-speech prediction}},
  author = {Toutanova, Kristina and Cherry, Colin},
  booktitle = {Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP},
  year = {2009},
  doi = {10.3115/1687878.1687947},
  url = {https://doi.org/10.3115/1687878.1687947}
}
```

- [← previous Unbounded Dependency Recovery for Parser Evaluation](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2009-rimell-unbounded-dependency-recovery-parser-evaluation/)
- [next → "cba to check the spelling": Investigating Parser Performance on Discussion Forum Posts](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2010-foster-cba-check-spelling-investigating-parser/)
- [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/)

## References

23 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Meni Adler, Yoav Goldberg, and Michael Elhadad. 2008. Unsupervised lexicon-based resolution of unknown words for full morpholological analysis. In Proceedings of ACL-08: HLT. [find](https://scholar.google.com/scholar?q=Meni+Adler%2C+Yoav+Goldberg%2C+and+Michael+Elhadad%2E+2008%2E+Unsupervised+lexicon%2Dbased+resolution+of+unknown+words+for+full+morpholological+analysis%2E+In+Proceedings+of+ACL%2D08%3A+HLT%2E)
2. Galen Andrew, Trond Grenager, and Christopher Manning. 2004. Verb sense and subcategorization: Using joint inference to improve performance on complementary tasks. In EMNLP. [find](https://scholar.google.com/scholar?q=Galen+Andrew%2C+Trond+Grenager%2C+and+Christopher+Manning%2E+2004%2E+Verb+sense+and+subcategorization%3A+Using+joint+inference+to+improve+performance+on+complementary+tasks%2E+In+EMNLP%2E)
3. Erwin Marsi Antal van den Bosch and Abdelhadi Soudi. 2007. Memory-based morphological analysis and part-of-speech tagging of arabic. In Abdelhadi Soudi, Antal van den Bosch, and Gunter Neumann, editors, Arabic Computational Morphology Knowledge-based and Empirical Methods. Springer. [find](https://scholar.google.com/scholar?q=Erwin+Marsi+Antal+van+den+Bosch+and+Abdelhadi+Soudi%2E+2007%2E+Memory%2Dbased+morphological+analysis+and+part%2Dof%2Dspeech+tagging+of+arabic%2E+In+Abdelhadi+Soudi%2C+Antal+van+den+Bosch%2C+and+Gunter+Neumann%2C+editor)
4. R. H. Baayen, R. Piepenbrock, and L. Gulikers. 1995. The CELEX lexical database. [find](https://scholar.google.com/scholar?q=R%2E+H%2E+Baayen%2C+R%2E+Piepenbrock%2C+and+L%2E+Gulikers%2E+1995%2E+The+CELEX+lexical+database%2E)
5. Antal Van Den Bosch and Walter Daelemans. Memory-based morphological analysis. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Antal+Van+Den+Bosch+and+Walter+Daelemans%2E+Memory%2Dbased+morphological+analysis%2E+In+Proceedings+of+the+37th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2E)
6. Alexander Clark. 2002. Memory-based learning of morphology with stochastic transducers. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), pages 513–520. [find](https://scholar.google.com/scholar?q=Alexander+Clark%2E+2002%2E+Memory%2Dbased+learning+of+morphology+with+stochastic+transducers%2E+In+Proceedings+of+the+40th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28ACL%29%2C+pages+513%E2%80%93520%2E)
7. Shay B. Cohen and Noah A. Smith. 2007. Joint morphological and syntactic disambiguation. In EMNLP. [find](https://scholar.google.com/scholar?q=Shay+B%2E+Cohen+and+Noah+A%2E+Smith%2E+2007%2E+Joint+morphological+and+syntactic+disambiguation%2E+In+EMNLP%2E)
8. Michael Collins. 2000. Discriminative reranking for natural language parsing. In ICML. [find](https://scholar.google.com/scholar?q=Michael+Collins%2E+2000%2E+Discriminative+reranking+for+natural+language+parsing%2E+In+ICML%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/collins-2000-discriminative-reranking-natural-language-parsing/)
9. M. Collins. 2002. Discriminative training methods for hidden markov models: Theory and experiments with perceptron algorithms. In EMNLP. [find](https://scholar.google.com/scholar?q=M%2E+Collins%2E+2002%2E+Discriminative+training+methods+for+hidden+markov+models%3A+Theory+and+experiments+with+perceptron+algorithms%2E+In+EMNLP%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/collins-2002-discriminative-training-methods-hidden-markov/)
10. S. Cucerzan and D. Yarowsky. 2000. Language independent minimally supervised induction of lexical probabilities. In Proceedings of ACL 2000. [find](https://scholar.google.com/scholar?q=S%2E+Cucerzan+and+D%2E+Yarowsky%2E+2000%2E+Language+independent+minimally+supervised+induction+of+lexical+probabilities%2E+In+Proceedings+of+ACL+2000%2E)
11. Markus Dreyer, Jason R. Smith, and Jason Eisner. 2008. Latent-variable modeling of string transductions with finite-state methods. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1080–1089, Honolulu, October. [find](https://scholar.google.com/scholar?q=Markus+Dreyer%2C+Jason+R%2E+Smith%2C+and+Jason+Eisner%2E+2008%2E+Latent%2Dvariable+modeling+of+string+transductions+with+finite%2Dstate+methods%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Lang)
12. Tomaž Erjavec and Saǎo Džeroski. 2004. Machine learning of morphosyntactic structure: lemmatizing unknown Slovene words. Applied Artificial Intelligence, 18:17— 41. [find](https://scholar.google.com/scholar?q=Tomaz%CC%8C+Erjavec+and+Saa%CC%8Co+Dz%CC%8Ceroski%2E+2004%2E+Machine+learning+of+morphosyntactic+structure%3A+lemmatizing+unknown+Slovene+words%2E+Applied+Artificial+Intelligence%2C+18%3A17%E2%80%94+41%2E)
13. Tomaž Erjavec. 2004. Multext-east version 3: Multilingual morphosyntactic specifications, lexicons and corpora. In Proceedings of LREC-04. [find](https://scholar.google.com/scholar?q=Tomaz%CC%8C+Erjavec%2E+2004%2E+Multext%2Deast+version+3%3A+Multilingual+morphosyntactic+specifications%2C+lexicons+and+corpora%2E+In+Proceedings+of+LREC%2D04%2E)
14. Jenny Rose Finkel, Christopher D. Manning, and Andrew Y. Ng. 2006. Solving the problem of cascading errors: Approximate bayesian inference for linguistic annotation pipelines. In EMNLP. [find](https://scholar.google.com/scholar?q=Jenny+Rose+Finkel%2C+Christopher+D%2E+Manning%2C+and+Andrew+Y%2E+Ng%2E+2006%2E+Solving+the+problem+of+cascading+errors%3A+Approximate+bayesian+inference+for+linguistic+annotation+pipelines%2E+In+EMNLP%2E)
15. Nizar Habash and Owen Rambow. 2005. Arabic tokenization, part-of-speech tagging and morphological disambiguation in one fell swoop. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Nizar+Habash+and+Owen+Rambow%2E+2005%2E+Arabic+tokenization%2C+part%2Dof%2Dspeech+tagging+and+morphological+disambiguation+in+one+fell+swoop%2E+In+Proceedings+of+the+43rd+Annual+Meeting+of+the+Association+for+Com) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2005-habash-arabic-tokenization-part-speech-tagging/)
16. Sittichai Jiampojamarn, Colin Cherry, and Grzegorz Kondrak. 2008. Joint processing and discriminative training for letter-to-phoneme conversion. In Proceedings of ACL-08: HLT, pages 905–913, Columbus, Ohio, June. [find](https://scholar.google.com/scholar?q=Sittichai+Jiampojamarn%2C+Colin+Cherry%2C+and+Grzegorz+Kondrak%2E+2008%2E+Joint+processing+and+discriminative+training+for+letter%2Dto%2Dphoneme+conversion%2E+In+Proceedings+of+ACL%2D08%3A+HLT%2C+pages+905%E2%80%93913%2C+Columbus%2C)
17. M. Marcus, B. Santorini, and Marcinkiewicz. 1993. Building a large annotated coprus of english: the penn treebank. Computational Linguistics, 19. [find](https://scholar.google.com/scholar?q=M%2E+Marcus%2C+B%2E+Santorini%2C+and+Marcinkiewicz%2E+1993%2E+Building+a+large+annotated+coprus+of+english%3A+the+penn+treebank%2E+Computational+Linguistics%2C+19%2E)
18. Raymond J. Mooney and Mary Elaine Califf. 1995. Induction of first-order decision lists: Results on learning the past tense of english verbs. Journal of Artificial Intelligence Research, 3:1—24. [find](https://scholar.google.com/scholar?q=Raymond+J%2E+Mooney+and+Mary+Elaine+Califf%2E+1995%2E+Induction+of+first%2Dorder+decision+lists%3A+Results+on+learning+the+past+tense+of+english+verbs%2E+Journal+of+Artificial+Intelligence+Research%2C+3%3A1%E2%80%9424%2E)
19. Eric Sven Ristad and Peter N. Yianilos. 1998. Learning string-edit distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(5):522–532. [find](https://scholar.google.com/scholar?q=Eric+Sven+Ristad+and+Peter+N%2E+Yianilos%2E+1998%2E+Learning+string%2Dedit+distance%2E+IEEE+Transactions+on+Pattern+Analysis+and+Machine+Intelligence%2C+20%285%29%3A522%E2%80%93532%2E)
20. Sunita Sarawagi and William Cohen. 2004. Semimarkov conditional random fields for information extraction. In ICML. 1999. [find](https://scholar.google.com/scholar?q=Sunita+Sarawagi+and+William+Cohen%2E+2004%2E+Semimarkov+conditional+random+fields+for+information+extraction%2E+In+ICML%2E+1999%2E)
21. Kristina Toutanova and Mark Johnson. 2008. A bayesian LDA-based model for semi-supervised part-of-speech tagging. In nips08. [find](https://scholar.google.com/scholar?q=Kristina+Toutanova+and+Mark+Johnson%2E+2008%2E+A+bayesian+LDA%2Dbased+model+for+semi%2Dsupervised+part%2Dof%2Dspeech+tagging%2E+In+nips08%2E)
22. Richard Wicentowski. 2002. Modeling and Learning Multilingual Inflectional Morphology in a Minimally Supervised Framework. Ph.D. thesis, Johns-Hopkins University. [find](https://scholar.google.com/scholar?q=Richard+Wicentowski%2E+2002%2E+Modeling+and+Learning+Multilingual+Inflectional+Morphology+in+a+Minimally+Supervised+Framework%2E+Ph%2ED%2E+thesis%2C+Johns%2DHopkins+University%2E)
23. R. Zens and H. Ney. 2004. Improvements in phrase-based statistical machine translation. In HLT-NAACL, pages 257–264, Boston, USA, May. [find](https://scholar.google.com/scholar?q=R%2E+Zens+and+H%2E+Ney%2E+2004%2E+Improvements+in+phrase%2Dbased+statistical+machine+translation%2E+In+HLT%2DNAACL%2C+pages+257%E2%80%93264%2C+Boston%2C+USA%2C+May%2E)