[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/merlo-2009-abstraction-generalisation-semantic-role-labels/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / Abstraction and Generalisation in Semantic Role Labels: PropBank, VerbNet or both?

# Abstraction and Generalisation in Semantic Role Labels: PropBank, VerbNet or both?

**Authors:** [Paola Merlo](https://md.mova.elamur.ai/free/en/docs/papers/authors/paola-merlo/), [Lonneke van der Plas](https://md.mova.elamur.ai/free/en/docs/papers/authors/lonneke-van-der-plas/) · **Year:** 2009 · **Venue:** Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP **Link:** [aclanthology.org/P09-1033](https://aclanthology.org/P09-1033/) · **Licence:** [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/) (ACL Anthology, before 2016) — summary only

[**14** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/merlo-2009-abstraction-generalisation-semantic-role-labels/) of the 140673 papers whose bibliographies we read[Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP](https://aclanthology.org/volumes/P09-1/)[licence](https://creativecommons.org/licenses/by-nc-sa/3.0/)

- [Publisher's page](https://aclanthology.org/P09-1033/)
- [PDF](https://aclanthology.org/P09-1033.pdf)

**Authors (2)**

1. Paola Merlo
2. Lonneke van der Plas

Citations and author metrics from OpenAlex (); authors with the same name are not always told apart.

## What it is about

The paper compares two role inventories, PropBank (abstract labels A0 to A5, AA, and adjunct tags) and VerbNet (23 thematic roles, grouped into 7 classes for the comparison). It does not train a parser or a labeller. It uses information-theoretic counts over the SemLink mapping of the Wall Street Journal training and development sections (106,459 role pairs; 68,268 dependencies for the syntax analysis). PropBank labels are easier to learn (entropy 1.37 bits, against 2.06 for the VerbNet groups). VerbNet labels carry more information and depend more on the verb: mutual information with the verb is 1.03 bits for VerbNet and 0.26 bits for PropBank. PropBank role sets correlate more with dependency labels (symmetric uncertainty 0.49, against 0.39). An animacy effect on the subject pronouns is significant for VerbNet only. The authors conclude that both inventories are useful: VerbNet when the verb is known, PropBank when it is not.

## Ideas

- I1. A two-tier role lookup in the frame reader: (verb, sense) gives a verb-specific thematic role; when the verb is not in the lexicon, fall back to a coarse abstract label (A0 or A1 style). Implement the order in `prag/src/roles.rs`, and measure accuracy at each step on school sentences.
- I2. Report the entropy, mutual information and symmetric uncertainty of Mova's own role labels against the verb and against the dependency relation, as a diagnostic table. It shows how much the rules depend on the verb, which the paper argues is the key property.
- I3. Test the animacy effect with a fixed pronoun list (I, you, he, she, it and so on) as a deterministic feature for agent assignment. The paper's test is a count-based ANOVA, which Mova can run with plain counts.
- I4. Use the role-set counts (PropBank: 62 role sets; VerbNet: 116 combinations) as a closed-world check: a proposition whose role set was never seen in the lessons is flagged rather than silently accepted. Measure the flag rate on held-out lessons.

## For Mova

The chapters are [13. Verbs](https://md.mova.elamur.ai/free/en/docs/languages/en/13-verbs/) and [28. Voice and the passive](https://md.mova.elamur.ai/free/en/docs/languages/en/28-voice-and-passive/). The code is `prag/src/roles.rs`, with the verb lexicon in `en/src/dict.rs`. Measure with the school test sentences and the two-fold `cv` split. Run twice and compare the outputs, since the counts must be ordered by BTreeMap.

Topics: [Semantic role labeling](https://md.mova.elamur.ai/free/en/docs/papers/topics/semantic-role-labeling/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work.

```
@inproceedings{merlo2009abstraction,
  title = {{Abstraction and Generalisation in Semantic Role Labels: PropBank, VerbNet or both?}},
  author = {Merlo, Paola and Plas, Lonneke van der},
  booktitle = {Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP},
  year = {2009},
  url = {https://aclanthology.org/P09-1033/}
}
```

- [← previous Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference](https://md.mova.elamur.ai/free/en/docs/papers/mccoy-2019-right-wrong-reasons-diagnosing-syntactic/)
- [next → Detecting Syntactic Change Using a Neural Part-of-Speech Tagger](https://md.mova.elamur.ai/free/en/docs/papers/merrill-2019-detecting-syntactic-change-neural-part/)
- [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/)

## References

21 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Judith Aissen. 2003. Differential object marking: Iconicity vs. economy. Natural Language and Linguistic Theory, 21:435–483. [find](https://scholar.google.com/scholar?q=Judith+Aissen%2E+2003%2E+Differential+object+marking%3A+Iconicity+vs%2E+economy%2E+Natural+Language+and+Linguistic+Theory%2C+21%3A435%E2%80%93483%2E)
2. Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley FrameNet project. In Proceedings of the Thirty-Sixth Annual Meeting of the Association for Computational Linguistics and Seventeenth International Conference on Computational Linguistics (ACL-COLING’98), pages 86–90, Montreal, Canada. [find](https://scholar.google.com/scholar?q=Collin+F%2E+Baker%2C+Charles+J%2E+Fillmore%2C+and+John+B%2E+Lowe%2E+1998%2E+The+Berkeley+FrameNet+project%2E+In+Proceedings+of+the+Thirty%2DSixth+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+Seve) [our card](https://md.mova.elamur.ai/free/en/docs/papers/lex-1998-baker-berkeley-framenet/)
3. David Dowty. 1991. Thematic proto-roles and argument selection. Language, 67(3):547–619. [find](https://scholar.google.com/scholar?q=David+Dowty%2E+1991%2E+Thematic+proto%2Droles+and+argument+selection%2E+Language%2C+67%283%29%3A547%E2%80%93619%2E)
4. Charles Fillmore. 1968. The case for case. In Emmon Bach and Harms, editors, Universals in Linguistic Theory, pages 1–88. Holt, Rinehart, and Winston. [find](https://scholar.google.com/scholar?q=Charles+Fillmore%2E+1968%2E+The+case+for+case%2E+In+Emmon+Bach+and+Harms%2C+editors%2C+Universals+in+Linguistic+Theory%2C+pages+1%E2%80%9388%2E+Holt%2C+Rinehart%2C+and+Winston%2E)
5. Jane Grimshaw. 1990. Argument Structure. MIT Press. [find](https://scholar.google.com/scholar?q=Jane+Grimshaw%2E+1990%2E+Argument+Structure%2E+MIT+Press%2E)
6. Jeffrey Gruber. 1965. Studies in Lexical Relation. MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=Jeffrey+Gruber%2E+1965%2E+Studies+in+Lexical+Relation%2E+MIT+Press%2C+Cambridge%2C+MA%2E)
7. Ray Jackendoff. 1972. Semantic Interpretation in Generative Grammar. MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=Ray+Jackendoff%2E+1972%2E+Semantic+Interpretation+in+Generative+Grammar%2E+MIT+Press%2C+Cambridge%2C+MA%2E)
8. Karin Kipper. 2005. VerbNet: A broad-coverage, comprehensive verb lexicon. Ph.D. thesis, University of Pennsylvania. [find](https://scholar.google.com/scholar?q=Karin+Kipper%2E+2005%2E+VerbNet%3A+A+broad%2Dcoverage%2C+comprehensive+verb+lexicon%2E+Ph%2ED%2E+thesis%2C+University+of+Pennsylvania%2E)
9. Beth Levin and Malka Rappaport Hovav. 2005. Argument Realization. Cambridge University Press, Cambridge, UK. [find](https://scholar.google.com/scholar?q=Beth+Levin+and+Malka+Rappaport+Hovav%2E+2005%2E+Argument+Realization%2E+Cambridge+University+Press%2C+Cambridge%2C+UK%2E)
10. Edward Loper, Szu ting Yi, and Martha Palmer. 2007. Combining lexical resources: Mapping between PropBank and VerbNet. In Proceedings of the IWCS. [find](https://scholar.google.com/scholar?q=Edward+Loper%2C+Szu+ting+Yi%2C+and+Martha+Palmer%2E+2007%2E+Combining+lexical+resources%3A+Mapping+between+PropBank+and+VerbNet%2E+In+Proceedings+of+the+IWCS%2E)
11. Christopher Manning and Hinrich Schuetze. 1999. Foundations of Statistical Natural Language Processing. MIT Press. [find](https://scholar.google.com/scholar?q=Christopher+Manning+and+Hinrich+Schuetze%2E+1999%2E+Foundations+of+Statistical+Natural+Language+Processing%2E+MIT+Press%2E)
12. Mitch Marcus, Beatrice Santorini, and M.A. Marcinkiewicz. 1993. Building a large annotated corpus of English: the Penn Treebank. Computational Linguistics, 19:313–330. [find](https://scholar.google.com/scholar?q=Mitch+Marcus%2C+Beatrice+Santorini%2C+and+M%2EA%2E+Marcinkiewicz%2E+1993%2E+Building+a+large+annotated+corpus+of+English%3A+the+Penn+Treebank%2E+Computational+Linguistics%2C+19%3A313%E2%80%93330%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/bank-1993-marcus-building-large-annotated-corpus-english/)
13. Paola Merlo and Gabriele Musillo. 2008. Semantic parsing for high-precision semantic role labelling. In Proceedings of the Twelfth Conference on Computational Natural Language Learning (CONLL-08), pages 1–8, Manchester, UK. [find](https://scholar.google.com/scholar?q=Paola+Merlo+and+Gabriele+Musillo%2E+2008%2E+Semantic+parsing+for+high%2Dprecision+semantic+role+labelling%2E+In+Proceedings+of+the+Twelfth+Conference+on+Computational+Natural+Language+Learning+%28CONLL%2D08%29%2C+pag)
14. Paola Merlo and Suzanne Stevenson. 2001. Automatic verb classification based on statistical distributions of argument structure. Computational Linguistics, 27(3):373–408. [find](https://scholar.google.com/scholar?q=Paola+Merlo+and+Suzanne+Stevenson%2E+2001%2E+Automatic+verb+classification+based+on+statistical+distributions+of+argument+structure%2E+Computational+Linguistics%2C+27%283%29%3A373%E2%80%93408%2E)
15. Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The Proposition Bank: An annotated corpus of semantic roles. Computational Linguistics, 31:71–105. [find](https://scholar.google.com/scholar?q=Martha+Palmer%2C+Daniel+Gildea%2C+and+Paul+Kingsbury%2E+2005%2E+The+Proposition+Bank%3A+An+annotated+corpus+of+semantic+roles%2E+Computational+Linguistics%2C+31%3A71%E2%80%93105%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/palmer-2005-proposition-bank-annotated-corpus-semantic/)
16. Josef Ruppenhofer, Michael Ellsworth, Miriam Petruck, Christopher Johnson, and Jan Scheffczyk. 2006. Framenet ii: Theory and practice. Technical report, Berkeley,CA. [find](https://scholar.google.com/scholar?q=Josef+Ruppenhofer%2C+Michael+Ellsworth%2C+Miriam+Petruck%2C+Christopher+Johnson%2C+and+Jan+Scheffczyk%2E+2006%2E+Framenet+ii%3A+Theory+and+practice%2E+Technical+report%2C+Berkeley%2CCA%2E)
17. Mihai Surdeanu, Richard Johansson, Adam Meyers, Lluı́s Màrquez, and Joakim Nivre. 2008. The CoNLL-2008 shared task on joint parsing of syntactic and semantic dependencies. In Proceedings of the 12th Conference on Computational Natural Language Learning (CoNLL-2008), pages 159–177. [find](https://scholar.google.com/scholar?q=Mihai+Surdeanu%2C+Richard+Johansson%2C+Adam+Meyers%2C+Llu%C4%B1%CC%81s+Ma%CC%80rquez%2C+and+Joakim+Nivre%2E+2008%2E+The+CoNLL%2D2008+shared+task+on+joint+parsing+of+syntactic+and+semantic+dependencies%2E+In+Proceedings+of+the+12th+) [our card](https://md.mova.elamur.ai/free/en/docs/papers/surdeanu-2008-conll-2008-shared-task-joint/)
18. Kristina Toutanova, Aria Haghighi, and Christopher D. Manning. 2008. A global joint model for semantic role labeling. Computational Linguistics, 34(2). [find](https://scholar.google.com/scholar?q=Kristina+Toutanova%2C+Aria+Haghighi%2C+and+Christopher+D%2E+Manning%2E+2008%2E+A+global+joint+model+for+semantic+role+labeling%2E+Computational+Linguistics%2C+34%282%29%2E)
19. Ian Witten and Eibe Frank. 2005. Data Mining. Elsevier. [find](https://scholar.google.com/scholar?q=Ian+Witten+and+Eibe+Frank%2E+2005%2E+Data+Mining%2E+Elsevier%2E)
20. Szu-ting Yi, Edward Loper, and Martha Palmer. 2007. Can semantic roles generalize across genres? In Proceedings of the Human Language Technologies 2007 (NAACL-HLT’07), pages 548–555, Rochester, New York, April. [find](https://scholar.google.com/scholar?q=Szu%2Dting+Yi%2C+Edward+Loper%2C+and+Martha+Palmer%2E+2007%2E+Can+semantic+roles+generalize+across+genres%3F+In+Proceedings+of+the+Human+Language+Technologies+2007+%28NAACL%2DHLT%E2%80%9907%29%2C+pages+548%E2%80%93555%2C+Rochester%2C+New+Yo)
21. Beñat Zapirain, Eneko Agirre, and Lluı́s Màrquez. 2008. Robustness and generalization of role sets: PropBank vs. VerbNet. In Proceedings of ACL-08: HLT, pages 550–558, Columbus, Ohio, June. [find](https://scholar.google.com/scholar?q=Ben%CC%83at+Zapirain%2C+Eneko+Agirre%2C+and+Llu%C4%B1%CC%81s+Ma%CC%80rquez%2E+2008%2E+Robustness+and+generalization+of+role+sets%3A+PropBank+vs%2E+VerbNet%2E+In+Proceedings+of+ACL%2D08%3A+HLT%2C+pages+550%E2%80%93558%2C+Columbus%2C+Ohio%2C+June%2E)