[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/lex-2008-rychly-logdice/) [home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / A Lexicographer-Friendly Association Score # A Lexicographer-Friendly Association Score **Authors:** [Pavel Rychlý](https://md.mova.elamur.ai/free/en/docs/papers/authors/pavel-rychlu/) · **Year:** 2008 · **Venue:** Proceedings of RASLAN 2008 (Recent Advances in Slavonic Natural Language Processing) **Link:** [nlp.fi.muni.cz/raslan/2008](https://nlp.fi.muni.cz/raslan/2008/) · **Licence:** Free access on the RASLAN site, no CC licence stated — summary only [**26** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/lex-2008-rychly-logdice/) of the 140673 papers whose bibliographies we readRASLAN 2008 (Recent Advances in Slavonic NLP), Masaryk University[licence](https://nlp.fi.muni.cz/raslan/2008/) - [Publisher's page](https://nlp.fi.muni.cz/raslan/2008/) - [PDF](https://nlp.fi.muni.cz/raslan/2008/papers/13.pdf) **Authors (1)** 1. Pavel Rychlý Citations and author metrics from OpenAlex (); authors with the same name are not always told apart. ## What it is about Association scores rank word pairs as collocation candidates, but most of them depend on corpus size and their values have no fixed meaning. Rychlý proposes logDice, a rescaled Dice coefficient: logDice = 14 + log2(2·f_xy / (f_x + f_y)). The maximum is 14, and a value near 0 means fewer than one co-occurrence per 16,000 occurrences of either word. A gain of 1 point means twice as frequent, and 7 points about 100 times. The paper compares T-score, MI, MI3, Dice, minimum sensitivity and MI × log frequency on the verb "break" in the British National Corpus. The Dice lists are the best, and logDice keeps their ranking while giving values that do not depend on corpus size. ## Ideas - I1. `en/src/lexgen.rs` or the lexicon build: compute logDice for candidate multiword pairs (adjacent and within a 5-token window) from an open corpus, and rank them. Measure precision at the top 100 against the expression base in `prag`, on a held-out split of it. - I2. Stability check: compute logDice on random 10%, 50% and 100% subsets of the school reading texts and compare the top 50 lists (overlap and rank change). The paper claims size-independence; Mova should measure it before using a fixed threshold. - I3. Threshold rule: use a fixed logDice cut-off (for example 0 as "no collocation") as a readable rule in `en` and report how many EWT verb-object pairs pass, with a short explanation per pair. ## For Mova The touch points are `en/src/lexgen.rs` and `en/src/dict.rs` (candidate lists) and the `prag` expression base for checking. The chapter is [19. The lexicon: vocabulary, multiword expressions and register](https://md.mova.elamur.ai/free/en/docs/languages/en/19-lexicon-mwe-register/). The paper's example data is the BNC, which is NC and may only be used for tests; build with open corpora such as EWT (CC BY-SA). Measure first the top-100 precision of logDice against the `prag` expression base, with two identical runs. ## Cite The paper itself, not this card. BibTeX from the library's record of the work. ``` @inproceedings{rychlu2008lexicographer, title = {{A Lexicographer-Friendly Association Score}}, author = {Rychlý, Pavel}, booktitle = {RASLAN 2008 (Recent Advances in Slavonic NLP), Masaryk University}, year = {2008}, url = {https://nlp.fi.muni.cz/raslan/2008/} } ``` - [← previous GDEX: Automatically Finding Good Dictionary Examples in a Corpus](https://md.mova.elamur.ai/free/en/docs/papers/lex-2008-kilgarriff-gdex-examples/) - [next → To Exhibit is not to Loiter: A Multilingual, Sense-Disambiguated Wiktionary for Measuring Verb Similarity](https://md.mova.elamur.ai/free/en/docs/papers/lex-2012-meyer-sense-disambiguated-wiktionary/) - [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/) ## References 5 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Smadja, F.: Retrieving Collocations from Text: Xtract. Computational Linguistics 19(1) (1994) 143–177. [find](https://scholar.google.com/scholar?q=Smadja%2C+F%2E%3A+Retrieving+Collocations+from+Text%3A+Xtract%2E+Computational+Linguistics+19%281%29+%281994%29+143%E2%80%93177%2E) 2. Kilgarriff, A., Rychlý, P., Smrž, P., Tugwell, D.: The Sketch Engine. Proceedings of Euralex (2004) 105–116. [find](https://scholar.google.com/scholar?q=Kilgarriff%2C+A%2E%2C+Rychl%C3%BD%2C+P%2E%2C+Smr%C5%BE%2C+P%2E%2C+Tugwell%2C+D%2E%3A+The+Sketch+Engine%2E+Proceedings+of+Euralex+%282004%29+105%E2%80%93116%2E) 3. Dice, L.: Measures of the Amount of Ecologic Association Between Species. Ecology 26(3) (1945) 297–302. [find](https://scholar.google.com/scholar?q=Dice%2C+L%2E%3A+Measures+of+the+Amount+of+Ecologic+Association+Between+Species%2E+Ecology+26%283%29+%281945%29+297%E2%80%93302%2E) 4. Evert, S.: The Statistics of Word Cooccurrences: Word Pairs and Collocations. Unpublished Ph.D. dissertation, University of Stuttgart (2004). [find](https://scholar.google.com/scholar?q=Evert%2C+S%2E%3A+The+Statistics+of+Word+Cooccurrences%3A+Word+Pairs+and+Collocations%2E+Unpublished+Ph%2ED%2E+dissertation%2C+University+of+Stuttgart+%282004%29%2E) 5. Rychlý, P.: Manatee/Bonito – A Modular Corpus Manager. In: P. Sojka, A. Horák (Eds.): RASLAN 2007 Proceedings (2007), pp. 65–70. 2 f xy f x + f y [find](https://scholar.google.com/scholar?q=Rychl%C3%BD%2C+P%2E%3A+Manatee%2FBonito+%E2%80%93+A+Modular+Corpus+Manager%2E+In%3A+P%2E+Sojka%2C+A%2E+Hor%C3%A1k+%28Eds%2E%29%3A+RASLAN+2007+Proceedings+%282007%29%2C+pp%2E+65%E2%80%9370%2E+2+f+xy+f+x+%2B+f+y)