[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/vilain-1995-model-theoretic-coreference-scoring-scheme/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / A Model-Theoretic Coreference Scoring Scheme

# A Model-Theoretic Coreference Scoring Scheme

**Authors:** Marc Vilain, John Burger, John Aberdeen, Dennis Connolly, Lynette Hirschman · **Year:** 1995 · **Venue:** Sixth Message Understanding Conference (MUC-6), Columbia, Maryland **Link:** [aclanthology.org/M95-1005](https://aclanthology.org/M95-1005/) (ACL Anthology M95-1005) **License of the paper:** [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/) (ACL Anthology, before 2016) — [creativecommons.org/licenses/by-nc-sa/3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/)

[**372** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/vilain-1995-model-theoretic-coreference-scoring-scheme/) of the 140673 papers whose bibliographies we read[Sixth Message Understanding Conference (MUC-6): Proceedings of a Conference Held in Columbia, Maryland, November 6-8, 1995](https://aclanthology.org/volumes/M95-1/)[licence](https://creativecommons.org/licenses/by-nc-sa/3.0/)

- [Publisher's page](https://aclanthology.org/M95-1005/)
- [PDF](https://aclanthology.org/M95-1005.pdf)

**Authors (5)**

1. Marc Vilain
2. John Burger
3. John Aberdeen
4. Dennis Connolly
5. Lynette Hirschman

Citations and author metrics from OpenAlex (); authors with the same name are not always told apart.

## Summary

The paper defines the scoring metric used for the coreference task of MUC-6, now known as the MUC metric. Coreference chains are viewed as equivalence classes, and scoring counts links rather than individual pairwise decisions. Recall for a key chain is the number of links needed to connect it minus the number of pieces the response splits it into, normalized by the minimal number of links; precision is computed symmetrically with key and response swapped. This makes the score independent of which particular links a system outputs, as long as the resulting partition is the same. The metric is simple and intuitive, but it is known to ignore singleton entities and to reward over-merging, which later metrics (B³, CEAF) were designed to address.

## How Mova uses it

- `coref/src/score.rs`: the MUC metric is implemented in Rust as one of the coreference scores, alongside B³, CEAF-e/CEAF-m, and CoNLL F1 (the average of MUC, B³ and CEAF-e).
- Mentions are matched by head word (as in the CRAC shared tasks), with and without singletons.
- The scorer is checked against the worked example of Pradhan et al. 2014 (MUC 0.40 / 0.40) and by negative controls: one giant cluster gives MUC recall 1 but CoNLL < 0.7; all-singletons gives MUC 0.

## Effectiveness in Mova

It is an evaluation metric, not a modelling idea, so there is no gain to measure. It is a component of the reported results, e.g. on GUM test (UD 2.18, gold trees, with singletons) the default sieve system scores CoNLL F1 73.04 with MUC F1 69.18, B³ 75.26, CEAF-e 74.68.

---

👨‍🔬💥

Topics: [Coreference resolution](https://md.mova.elamur.ai/free/en/docs/papers/topics/coreference-resolution/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work.

```
@inproceedings{vilain1995model,
  title = {{A Model-Theoretic Coreference Scoring Scheme}},
  author = {Vilain, Marc and Burger, John and Aberdeen, John and Connolly, Dennis and Hirschman, Lynette},
  booktitle = {Sixth Message Understanding Conference (MUC-6): Proceedings of a Conference Held in Columbia, Maryland, November 6-8, 1995},
  year = {1995},
  url = {https://aclanthology.org/M95-1005/}
}
```

- [← previous A New English Grammar, Logical and Historical](https://md.mova.elamur.ai/free/en/docs/papers/sweet-1891-new-english-grammar-logical/)
- [next → BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://md.mova.elamur.ai/free/en/docs/papers/warstadt-2020-blimp-benchmark-linguistic-minimal-pairs/)
- [↑ back to papers](https://md.mova.elamur.ai/free/en/docs/papers/)