[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/recasens-2010-coreference-resolution-across-corpora-languages/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / Coreference Resolution across Corpora: Languages, Coding Schemes, and Preprocessing Information

# Coreference Resolution across Corpora: Languages, Coding Schemes, and Preprocessing Information

**Authors:** [Marta Recasens](https://md.mova.elamur.ai/free/en/docs/papers/authors/marta-recasens/), [Eduard Hovy](https://md.mova.elamur.ai/free/en/docs/papers/authors/eduard-hovy/) · **Year:** 2010 · **Venue:** Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL 2010) **Link:** [aclanthology.org/P10-1144](https://aclanthology.org/P10-1144/) · **Licence:** [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/) (ACL Anthology, before 2016) — summary only

[**25** citations](https://openalex.org/works?filter=cites:W2155347856) openalex-snapshot · 2026-10-14[**15** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/recasens-2010-coreference-resolution-across-corpora-languages/) of the 140673 papers whose bibliographies we read[Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics](https://aclanthology.org/volumes/P10-1/)[licence](https://creativecommons.org/licenses/by-nc-sa/3.0/)

- [Publisher's page](https://aclanthology.org/P10-1144/)
- [PDF](https://aclanthology.org/P10-1144.pdf)
- [OpenAlex](https://openalex.org/W2155347856)

**Authors (2)**

1. [Marta Recasens](https://openalex.org/A5103978745) h 19 · 2086 citations · 72 works
2. [Eduard Hovy](https://openalex.org/A5060225743) [ORCID](https://orcid.org/0000-0002-3270-7903) · h 93 · 43467 citations · 690 works

Citations and author metrics from openalex-snapshot (2026-10-14); authors with the same name are not always told apart.

## What it is about

The paper shows that a coreference score says as much about the corpus as about the system. The same entity-based system, CISTELL, is trained and tested under three corpus choices: language (English against Spanish, using OntoNotes and AnCora, both news), coding scheme (ACE against OntoNotes on the same 100,000-word TDT portion), and preprocessing (gold against automatic parses). The system uses four basket-growing strategies (head match, strong match, best match, weak match) and a memory-based classifier. The scores are MUC, B3 and CEAF. The rankings of the strategies disagree across the metrics: MUC rewards weak match most, B3 rewards head match most, and CEAF is split between super-strong and best match. In both datasets about 85% of entities are singletons and 15% have two or more mentions, with 3.9 mentions per multi-mention entity in AnCora and 3.5 in OntoNotes. The authors conclude that system comparison is only meaningful when corpus parameters match exactly.

## Ideas

- I1. Run Mova's sieve on the same text with two annotation schemes (for example, with and without appositives and predicate nominals as coreferent) and report the change in each score. A change that appears only under one scheme is a scheme effect, not a system effect.
- I2. Put the metric disagreement into `coref/src/eval.rs`: a table of MUC, B3 and CEAF for each variant of the sieve, and a rule that a claim of improvement needs the same ranking in at least two of the three metrics.
- I3. Mention-type table for Spanish-English contrast: the paper shows zero subjects change the mention distribution. Mova's English-only setting can measure the same kind of shift on a set of sentences with pronoun-dropped and full-subject variants, as a check of the pronoun rules in [12. Pronouns, determiners and numerals](https://md.mova.elamur.ai/free/en/docs/languages/en/12-pronouns-determiners-numerals/).

## For Mova

Code: `coref/src/eval.rs` (three metrics side by side), `coref/src/sieve.rs` (variants as separate configurations), `coref/src/corefud.rs` (scheme conversion). Chapter: [35. Reference and coreference](https://md.mova.elamur.ai/free/en/docs/languages/en/35-reference-and-coreference/). Measure: the sieve under two scheme settings and three metrics, with the rankings printed; run twice and compare the output files byte for byte.

Topics: [Coreference resolution](https://md.mova.elamur.ai/free/en/docs/papers/topics/coreference-resolution/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work.

```
@inproceedings{recasens2010coreference,
  title = {{Coreference Resolution across Corpora: Languages, Coding Schemes, and Preprocessing Information}},
  author = {Recasens, Marta and Hovy, Eduard},
  booktitle = {Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics},
  year = {2010},
  url = {https://aclanthology.org/P10-1144/}
}
```

- [← previous EQUATE: A Benchmark Evaluation Framework for Quantitative Reasoning in Natural Language Inference](https://md.mova.elamur.ai/free/en/docs/papers/ravichander-2019-equate-benchmark-evaluation-framework-quantitative/)
- [next → A Lexical Process Model of Nominal Compounding in English](https://md.mova.elamur.ai/free/en/docs/papers/rhyne-1975-lexical-process-model-nominal-compounding/)
- [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/)

## References

30 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Amit Bagga and Breck Baldwin. 1998. Algorithms for scoring coreference chains. In Proceedings of the LREC 1998 Workshop on Linguistic Coreference, pages 563–566, Granada, Spain. [find](https://scholar.google.com/scholar?q=Amit+Bagga+and+Breck+Baldwin%2E+1998%2E+Algorithms+for+scoring+coreference+chains%2E+In+Proceedings+of+the+LREC+1998+Workshop+on+Linguistic+Coreference%2C+pages+563%E2%80%93566%2C+Granada%2C+Spain%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/bagga-1998-algorithms-scoring-coreference-chains/)
2. Eric Bengtson and Dan Roth. 2008. Understanding the value of features for coreference resolution. In Proceedings of EMNLP 2008, pages 294–303, Honolulu, Hawaii. [find](https://scholar.google.com/scholar?q=Eric+Bengtson+and+Dan+Roth%2E+2008%2E+Understanding+the+value+of+features+for+coreference+resolution%2E+In+Proceedings+of+EMNLP+2008%2C+pages+294%E2%80%93303%2C+Honolulu%2C+Hawaii%2E)
3. Thorsten Brants. 2000. TnT – A statistical part-of-speech tagger. In Proceedings of ANLP 2000, Seattle, WA. [↗](https://doi.org/10.48550/arxiv.cs/0003055) [our card](https://md.mova.elamur.ai/free/en/docs/papers/brants-2000-tnt-statistical-part-speech-tagger/)
4. Aron Culotta, Michael Wick, Robert Hall, and Andrew McCallum. 2007. First-order probabilistic models for coreference resolution. In Proceedings of HLT-NAACL 2007, pages 81–88, Rochester, New York. [find](https://scholar.google.com/scholar?q=Aron+Culotta%2C+Michael+Wick%2C+Robert+Hall%2C+and+Andrew+McCallum%2E+2007%2E+First%2Dorder+probabilistic+models+for+coreference+resolution%2E+In+Proceedings+of+HLT%2DNAACL+2007%2C+pages+81%E2%80%9388%2C+Rochester%2C+New+York%2E)
5. Walter Daelemans and Antal Van den Bosch. 2005. Memory-Based Language Processing. Cambridge University Press. [↗](https://doi.org/10.1017/cbo9780511486579)
6. Pascal Denis and Jason Baldridge. 2009. Global joint models for coreference resolution and named entity classification. Procesamiento del Lenguaje Natural, 42:87–96. [find](https://scholar.google.com/scholar?q=Pascal+Denis+and+Jason+Baldridge%2E+2009%2E+Global+joint+models+for+coreference+resolution+and+named+entity+classification%2E+Procesamiento+del+Lenguaje+Natural%2C+42%3A87%E2%80%9396%2E)
7. George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie Strassel, and Ralph Weischedel. 2004. The Automatic Content Extraction (ACE) Program - Tasks, Data, and Evaluation. In Proceedings of LREC 2004, pages 837–840. [find](https://scholar.google.com/scholar?q=George+Doddington%2C+Alexis+Mitchell%2C+Mark+Przybocki%2C+Lance+Ramshaw%2C+Stephanie+Strassel%2C+and+Ralph+Weischedel%2E+2004%2E+The+Automatic+Content+Extraction+%28ACE%29+Program+%2D+Tasks%2C+Data%2C+and+Evaluation%2E+In+Proc)
8. Christiane Fellbaum. 1998. WordNet: An Electronic Lexical Database. The MIT Press. [find](https://scholar.google.com/scholar?q=Christiane+Fellbaum%2E+1998%2E+WordNet%3A+An+Electronic+Lexical+Database%2E+The+MIT+Press%2E)
9. Jenny Rose Finkel and Christopher D. Manning. 2008. Enforcing transitivity in coreference resolution. In Proceedings of ACL-HLT 2008, pages 45– 48, Columbus, Ohio. [find](https://scholar.google.com/scholar?q=Jenny+Rose+Finkel+and+Christopher+D%2E+Manning%2E+2008%2E+Enforcing+transitivity+in+coreference+resolution%2E+In+Proceedings+of+ACL%2DHLT+2008%2C+pages+45%E2%80%93+48%2C+Columbus%2C+Ohio%2E)
10. Aria Haghighi and Dan Klein. 2009. Simple coreference resolution with rich syntactic and semantic features. In Proceedings of EMNLP 2009, pages 1152–1161, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Aria+Haghighi+and+Dan+Klein%2E+2009%2E+Simple+coreference+resolution+with+rich+syntactic+and+semantic+features%2E+In+Proceedings+of+EMNLP+2009%2C+pages+1152%E2%80%931161%2C+Singapore%2E+Association+for+Computational+Ling) [our card](https://md.mova.elamur.ai/free/en/docs/papers/haghighi-2009-simple-coreference-resolution-rich-syntactic/)
11. Johan Hall, Jens Nilsson, Joakim Nivre, Gülsen Eryigit, Beáta Megyesi, Mattias Nilsson, and Markus Saers. 2007. Single malt or blended? A study in multilingual parser optimization. In Proceedings of the CoNLL shared task session of EMNLP-CoNLL 2007, pages 933–939. [find](https://scholar.google.com/scholar?q=Johan+Hall%2C+Jens+Nilsson%2C+Joakim+Nivre%2C+Gu%CC%88lsen+Eryigit%2C+Bea%CC%81ta+Megyesi%2C+Mattias+Nilsson%2C+and+Markus+Saers%2E+2007%2E+Single+malt+or+blended%3F+A+study+in+multilingual+parser+optimization%2E+In+Proceedings+of)
12. Lynette Hirschman and Nancy Chinchor. 1997. MUC-7 Coreference Task Definition – Version 3.0. In Proceedings of MUC-7. [find](https://scholar.google.com/scholar?q=Lynette+Hirschman+and+Nancy+Chinchor%2E+1997%2E+MUC%2D7+Coreference+Task+Definition+%E2%80%93+Version+3%2E0%2E+In+Proceedings+of+MUC%2D7%2E)
13. Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. OntoNotes: the 90% solution. In Proceedings of HLT-NAACL 2006, pages 57–60. [find](https://scholar.google.com/scholar?q=Eduard+Hovy%2C+Mitchell+Marcus%2C+Martha+Palmer%2C+Lance+Ramshaw%2C+and+Ralph+Weischedel%2E+2006%2E+OntoNotes%3A+the+90%25+solution%2E+In+Proceedings+of+HLT%2DNAACL+2006%2C+pages+57%E2%80%9360%2E)
14. Taku Kudoh and Yuji Matsumoto. 2000. Use of support vector learning for chunk identification. In Proceedings of CoNLL 2000 and LLL 2000, pages 142– 144, Lisbon, Portugal. [find](https://scholar.google.com/scholar?q=Taku+Kudoh+and+Yuji+Matsumoto%2E+2000%2E+Use+of+support+vector+learning+for+chunk+identification%2E+In+Proceedings+of+CoNLL+2000+and+LLL+2000%2C+pages+142%E2%80%93+144%2C+Lisbon%2C+Portugal%2E)
15. Xiaoqiang Luo and Imed Zitouni. 2005. Multi-lingual coreference resolution with syntactic features. In Proceedings of HLT-EMNLP 2005, pages 660–667, Vancouver. [find](https://scholar.google.com/scholar?q=Xiaoqiang+Luo+and+Imed+Zitouni%2E+2005%2E+Multi%2Dlingual+coreference+resolution+with+syntactic+features%2E+In+Proceedings+of+HLT%2DEMNLP+2005%2C+pages+660%E2%80%93667%2C+Vancouver%2E)
16. Xiaoqiang Luo, Abe Ittycheriah, Hongyan Jing, Nanda Kambhatla, and Salim Roukos. 2004. A mention-synchronous coreference resolution algorithm based on the Bell tree. In Proceedings of ACL 2004, pages 21–26, Barcelona. [find](https://scholar.google.com/scholar?q=Xiaoqiang+Luo%2C+Abe+Ittycheriah%2C+Hongyan+Jing%2C+Nanda+Kambhatla%2C+and+Salim+Roukos%2E+2004%2E+A+mention%2Dsynchronous+coreference+resolution+algorithm+based+on+the+Bell+tree%2E+In+Proceedings+of+ACL+2004%2C+pages+)
17. Xiaoqiang Luo. 2005. On coreference resolution performance metrics. In Proceedings of HLT-EMNLP 2005, pages 25–32, Vancouver. [find](https://scholar.google.com/scholar?q=Xiaoqiang+Luo%2E+2005%2E+On+coreference+resolution+performance+metrics%2E+In+Proceedings+of+HLT%2DEMNLP+2005%2C+pages+25%E2%80%9332%2C+Vancouver%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/luo-2005-coreference-resolution-performance-metrics/)
18. Vincent Ng and Claire Cardie. 2002. Improving machine learning approaches to coreference resolution. In Proceedings of ACL 2002, pages 104–111, Philadelphia. [find](https://scholar.google.com/scholar?q=Vincent+Ng+and+Claire+Cardie%2E+2002%2E+Improving+machine+learning+approaches+to+coreference+resolution%2E+In+Proceedings+of+ACL+2002%2C+pages+104%E2%80%93111%2C+Philadelphia%2E)
19. Vincent Ng. 2009. Graph-cut-based anaphoricity determination for coreference resolution. In Proceedings of NAACL-HLT 2009, pages 575–583, Boulder, Colorado. [find](https://scholar.google.com/scholar?q=Vincent+Ng%2E+2009%2E+Graph%2Dcut%2Dbased+anaphoricity+determination+for+coreference+resolution%2E+In+Proceedings+of+NAACL%2DHLT+2009%2C+pages+575%E2%80%93583%2C+Boulder%2C+Colorado%2E)
20. Hoifung Poon and Pedro Domingos. 2008. Joint unsupervised coreference resolution with Markov logic. In Proceedings of EMNLP 2008, pages 650–659, Honolulu, Hawaii. [find](https://scholar.google.com/scholar?q=Hoifung+Poon+and+Pedro+Domingos%2E+2008%2E+Joint+unsupervised+coreference+resolution+with+Markov+logic%2E+In+Proceedings+of+EMNLP+2008%2C+pages+650%E2%80%93659%2C+Honolulu%2C+Hawaii%2E)
21. Sameer S. Pradhan, Eduard Hovy, Mitch Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2007. Ontonotes: A unified relational semantic representation. In Proceedings of ICSC 2007, pages 517–526, Washington, DC. [↗](https://doi.org/10.1142/s1793351x07000251)
22. Marta Recasens and Eduard Hovy. 2009. A Deeper Look into Features for Coreference Resolution. In S. Lalitha Devi, A. Branco, and R. Mitkov, editors, Anaphora Processing and Applications (DAARC 2009), volume 5847 of LNAI, pages 29–42. Springer-Verlag. [find](https://scholar.google.com/scholar?q=Marta+Recasens+and+Eduard+Hovy%2E+2009%2E+A+Deeper+Look+into+Features+for+Coreference+Resolution%2E+In+S%2E+Lalitha+Devi%2C+A%2E+Branco%2C+and+R%2E+Mitkov%2C+editors%2C+Anaphora+Processing+and+Applications+%28DAARC+2009%29%2C+)
23. Marta Recasens and M. Antònia Martı́. 2009. AnCora-CO: Coreferentially annotated corpora for Spanish and Catalan. Language Resources and Evaluation, DOI 10.1007/s10579-009-9108-x. [↗](https://doi.org/10.1007/s10579-009-9108-x)
24. Marta Recasens, Lluı́s Màrquez, Emili Sapena, M. Antònia Martı́, Mariona Taulé, Véronique Hoste, Massimo Poesio, and Yannick Versley. 2010. SemEval-2010 Task 1: Coreference resolution in multiple languages. In Proceedings of the Fifth International Workshop on Semantic Evaluations (SemEval 2010), Uppsala, Sweden. [find](https://scholar.google.com/scholar?q=Marta+Recasens%2C+Llu%C4%B1%CC%81s+Ma%CC%80rquez%2C+Emili+Sapena%2C+M%2E+Anto%CC%80nia+Mart%C4%B1%CC%81%2C+Mariona+Taule%CC%81%2C+Ve%CC%81ronique+Hoste%2C+Massimo+Poesio%2C+and+Yannick+Versley%2E+2010%2E+SemEval%2D2010+Task+1%3A+Coreference+resolution+in+multiple+)
25. Wee M. Soon, Hwee T. Ng, and Daniel C. Y. Lim. 2001. A machine learning approach to coreference resolution of noun phrases. Computational Linguistics, 27(4):521–544. [↗](https://doi.org/10.1162/089120101753342653) [our card](https://md.mova.elamur.ai/free/en/docs/papers/soon-2001-machine-learning-approach-coreference-resolution/)
26. Veselin Stoyanov, Nathan Gilbert, Claire Cardie, and Ellen Riloff. 2009. Conundrums in noun phrase coreference resolution: Making sense of the state-of-the-art. In Proceedings of ACL-IJCNLP 2009, pages 656–664, Singapore. [find](https://scholar.google.com/scholar?q=Veselin+Stoyanov%2C+Nathan+Gilbert%2C+Claire+Cardie%2C+and+Ellen+Riloff%2E+2009%2E+Conundrums+in+noun+phrase+coreference+resolution%3A+Making+sense+of+the+state%2Dof%2Dthe%2Dart%2E+In+Proceedings+of+ACL%2DIJCNLP+2009%2C+page) [our card](https://md.mova.elamur.ai/free/en/docs/papers/stoyanov-2009-conundrums-noun-phrase-coreference-resolution/)
27. Erik F. Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 Shared Task: Language-independent Named Entity Recognition. In Walter Daelemans and Miles Osborne, editors, Proceedings of CoNLL 2003, pages 142–147. Edmonton, Canada. [↗](https://doi.org/10.48550/arxiv.cs/0306050)
28. Olga Uryupina. 2006. Coreference resolution with and without linguistic knowledge. In Proceedings of LREC 2006. [find](https://scholar.google.com/scholar?q=Olga+Uryupina%2E+2006%2E+Coreference+resolution+with+and+without+linguistic+knowledge%2E+In+Proceedings+of+LREC+2006%2E)
29. Kees van Deemter and Rodger Kibble. 2000. On coreferring: Coreference in MUC and related annotation schemes. Computational Linguistics, 26(4):629– 637. [↗](https://doi.org/10.1162/089120100750105966) [our card](https://md.mova.elamur.ai/free/en/docs/papers/deemter-2000-coreferring-coreference-muc-related-annotation/)
30. Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model-theoretic coreference scoring scheme. In Proceedings of MUC-6, pages 45–52, San Francisco. [find](https://scholar.google.com/scholar?q=Marc+Vilain%2C+John+Burger%2C+John+Aberdeen%2C+Dennis+Connolly%2C+and+Lynette+Hirschman%2E+1995%2E+A+model%2Dtheoretic+coreference+scoring+scheme%2E+In+Proceedings+of+MUC%2D6%2C+pages+45%E2%80%9352%2C+San+Francisco%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/vilain-1995-model-theoretic-coreference-scoring-scheme/)