[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/aclbulk-2021-stekel-word-sense-induction-attentive-context/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [Ancient Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/hbo/) / Word Sense Induction with Attentive Context Clustering

# Word Sense Induction with Attentive Context Clustering

**Authors:** Moshe Stekel; [Amos Azaria](https://md.mova.elamur.ai/free/en/docs/papers/authors/amos-azaria/); [Shai Gordin](https://md.mova.elamur.ai/free/en/docs/papers/authors/shai-gordin/) · **Year:** 2021 · **Venue:** Proceedings of the Workshop on Natural Language Processing for Digital Humanities **Link:** [aclanthology.org/2021.nlp4dh-1.17](https://aclanthology.org/2021.nlp4dh-1.17/) · library `aclbulk-2021-stekel-word-sense-induction-attentive-context` · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)

[**3** citations](https://openalex.org/works?filter=cites:W4292617290) openalex · 2026-10-18[**2** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/aclbulk-2021-stekel-word-sense-induction-attentive-context/) of the 140673 papers whose bibliographies we read[Proceedings of the Workshop on Natural Language Processing for Digital Humanities](https://aclanthology.org/volumes/2021.nlp4dh-1/)[licence](https://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://aclanthology.org/2021.nlp4dh-1.17/)
- [PDF](https://aclanthology.org/2021.nlp4dh-1.17.pdf)
- [DOI 10.46298/jdmdh.9175](https://doi.org/10.46298/jdmdh.9175)
- [OpenAlex](https://openalex.org/W4292617290)

**Authors (3)**

1. [Moshe Stekel](https://openalex.org/A5029003367) h 1 · 3 citations · 2 works
2. [Amos Azaria](https://openalex.org/A5082567694) [ORCID](https://orcid.org/0000-0002-5057-1309) · h 23 · 2684 citations · 158 works
3. [Shai Gordin](https://openalex.org/A5068149720) [ORCID](https://orcid.org/0000-0002-8359-382X) · h 8 · 261 citations · 60 works

Citations and author metrics from openalex (2026-10-18); authors with the same name are not always told apart.

## What it is about

ACCWSI induces the senses of an ambiguous lemma without a large pretrained model. A small word2vec model is trained on the corpus; for each sentence with the query lemma, cosine similarity between the lemma vector and each context word gives softmax weights, and the weighted sum is a context-aware vector. These vectors are clustered with DBSCAN, and eps is chosen by shrinking it until more than half the points are noise. On SemEval-2010 Task 14 the method reaches a supervised recall of 63.7% (all words), and 62.7% when trained on 2.6% of the data, against 62.4% for the best listed competitor. On Biblical Hebrew it separates two senses of khalal (dead body, desecrate) and five of zakar (male, memory). The word2vec window is 5 for English and 2 for Classical Hebrew.

## Ideas

- I1. Set the word2vec window per language from a small gold check, not a constant: the authors found 2 for Classical Hebrew against 5 for English, which fits the short verse structure.
- I2. Keep the attention highlight as the explanation: the context words with the largest weights for each cluster say why a use was put in that sense. This gives a transparent sense label that `world` can store with each frame.
- I3. Build a small hand-checked gold set from the paper's own examples (khalal, zakar, and the "bank" test) and use it as a regression test for any sense-splitting rule, with a negative control (a shuffled-context run that must fail).
- I4. Use their eps rule (tighten until noise passes 50%) as a deterministic splitter, with BTreeMap ordering so the cut is reproducible.

## For Mova

I1 and I2 apply to the lemma senses in [The lexicon](https://md.mova.elamur.ai/free/en/docs/languages/hbo/10-lexicon/), and to `world` for sense frames. I3 is a test for the same. I4 is a candidate rule for `macula dict` lemma senses. The method is embedding-based, so it is a teacher-side or diagnostic tool, not part of the parser.

Topics: [Word sense disambiguation](https://md.mova.elamur.ai/free/en/docs/papers/topics/word-sense-disambiguation/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.46298/jdmdh.9175](https://doi.org/10.46298/jdmdh.9175)

```
@inproceedings{stekel2021word,
  title = {{Word Sense Induction with Attentive Context Clustering}},
  author = {Stekel, Moshe and Azaria, Amos and Gordin, Shai},
  booktitle = {Proceedings of the Workshop on Natural Language Processing for Digital Humanities},
  year = {2021},
  doi = {10.46298/jdmdh.9175},
  url = {https://doi.org/10.46298/jdmdh.9175}
}
```

- [← previous Transliteration of Judeo-Arabic Texts into Arabic Script Using Recurrent Neural Networks](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2020-terner-transliteration-judeo-arabic-texts-into/)
- [next → Handling Stress in Finite-State Morphological Analyzers for Ancient Greek and Ancient Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2022-swanson-handling-stress-finite-state-morphological/)
- [↑ back to Ancient Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/hbo/)

## References

19 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Eneko Agirre and German Rigau. 1996. Word sense disambiguation using conceptual density. arXiv preprint cmp-lg/9606007. [find](https://scholar.google.com/scholar?q=Eneko+Agirre+and+German+Rigau%2E+1996%2E+Word+sense+disambiguation+using+conceptual+density%2E+arXiv+preprint+cmp%2Dlg%2F9606007%2E)
2. Asaf Amrami and Yoav Goldberg. 2019. Towards better substitution-based word sense induction. arXiv preprint arXiv:1905.12598. [↗](https://arxiv.org/abs/1905.12598)
3. Stefan Bordag. 2006. Word sense induction: Triplet-based clustering and automatic evaluation. In 11th Conference of the European Chapter of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Stefan+Bordag%2E+2006%2E+Word+sense+induction%3A+Triplet%2Dbased+clustering+and+automatic+evaluation%2E+In+11th+Conference+of+the+European+Chapter+of+the+Association+for+Computational+Linguistics%2E)
4. Baobao Chang, Wenzhe Pei, and Miaohong Chen. 2014. Inducing word sense with automatically learned hidden concepts. In Proceedings of COL-ING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 355–364. [find](https://scholar.google.com/scholar?q=Baobao+Chang%2C+Wenzhe+Pei%2C+and+Miaohong+Chen%2E+2014%2E+Inducing+word+sense+with+automatically+learned+hidden+concepts%2E+In+Proceedings+of+COL%2DING+2014%2C+the+25th+International+Conference+on+Computational+Li)
5. Jim Cowie, Joe Guthrie, and Louise Guthrie. 1992. Lexical disambiguation using simulated annealing. In COLING 1992 Volume 1: The 14th International Conference on Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jim+Cowie%2C+Joe+Guthrie%2C+and+Louise+Guthrie%2E+1992%2E+Lexical+disambiguation+using+simulated+annealing%2E+In+COLING+1992+Volume+1%3A+The+14th+International+Conference+on+Computational+Linguistics%2E)
6. Matan Eyal, Shoval Sadde, Hillel Taub-Tabib, and Yoav Goldberg. 2021. Large scale substitution-based word sense induction. arXiv preprint arXiv:2110.07681. [↗](https://arxiv.org/abs/2110.07681) [our card](https://md.mova.elamur.ai/free/en/docs/papers/lex-2022-eyal-substitution-wsi/)
7. Aysu Ezen-Can. 2020. A comparison of lstm and bert for small corpus. arXiv preprint arXiv:2009.05451. [↗](https://arxiv.org/abs/2009.05451)
8. Evelina Fedorenko and Rosemary Varley. 2016. Language and thought are not the same thing: evidence from neuroimaging and neurological patients. Annals of the New York Academy of Sciences, 1369(1):132. [find](https://scholar.google.com/scholar?q=Evelina+Fedorenko+and+Rosemary+Varley%2E+2016%2E+Language+and+thought+are+not+the+same+thing%3A+evidence+from+neuroimaging+and+neurological+patients%2E+Annals+of+the+New+York+Academy+of+Sciences%2C+1369%281%29%3A132%2E)
9. Andrea Galassi, Marco Lippi, and Paolo Torroni. 2020. Attention in natural language processing. IEEE Transactions on Neural Networks and Learning Systems. [find](https://scholar.google.com/scholar?q=Andrea+Galassi%2C+Marco+Lippi%2C+and+Paolo+Torroni%2E+2020%2E+Attention+in+natural+language+processing%2E+IEEE+Transactions+on+Neural+Networks+and+Learning+Systems%2E)
10. Yoav Goldberg and Omer Levy. 2014. word2vec explained: deriving mikolov et al.’s negative-sampling word-embedding method. arXiv preprint arXiv:1402.3722. [↗](https://arxiv.org/abs/1402.3722)
11. Greg Hamerly and Charles Elkan. 2004. Learning the k in k-means. Advances in neural information processing systems, 16:281–288. [find](https://scholar.google.com/scholar?q=Greg+Hamerly+and+Charles+Elkan%2E+2004%2E+Learning+the+k+in+k%2Dmeans%2E+Advances+in+neural+information+processing+systems%2C+16%3A281%E2%80%93288%2E)
12. Dekang Lin. 1998. Automatic retrieval and clustering of similar words. In 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 2, pages 768–774. [find](https://scholar.google.com/scholar?q=Dekang+Lin%2E+1998%2E+Automatic+retrieval+and+clustering+of+similar+words%2E+In+36th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+17th+International+Conference+on+Computational+Lingui)
13. Suresh Manandhar, Ioannis Klapaftis, Dmitriy Dligach, and Sameer Pradhan. 2010. Semeval-2010 task 14: Word sense induction &disambiguation. In Proceedings of the 5th international workshop on semantic evaluation, pages 63–68. [find](https://scholar.google.com/scholar?q=Suresh+Manandhar%2C+Ioannis+Klapaftis%2C+Dmitriy+Dligach%2C+and+Sameer+Pradhan%2E+2010%2E+Semeval%2D2010+task+14%3A+Word+sense+induction+%26disambiguation%2E+In+Proceedings+of+the+5th+international+workshop+on+semantic)
14. Patrick Pantel and Dekang Lin. 2002. Discovering word senses from text. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 613–619. [find](https://scholar.google.com/scholar?q=Patrick+Pantel+and+Dekang+Lin%2E+2002%2E+Discovering+word+senses+from+text%2E+In+Proceedings+of+the+eighth+ACM+SIGKDD+international+conference+on+Knowledge+discovery+and+data+mining%2C+pages+613%E2%80%93619%2E)
15. Douglas A Reynolds. 2009. Gaussian mixture models. Encyclopedia of biometrics, 741:659–663. [find](https://scholar.google.com/scholar?q=Douglas+A+Reynolds%2E+2009%2E+Gaussian+mixture+models%2E+Encyclopedia+of+biometrics%2C+741%3A659%E2%80%93663%2E)
16. Erich Schubert, Jörg Sander, Martin Ester, Hans Peter Kriegel, and Xiaowei Xu. 2017. Dbscan revisited, revisited: why and how you should (still) use dbscan. ACM Transactions on Database Systems (TODS), 42(3):1–21. [find](https://scholar.google.com/scholar?q=Erich+Schubert%2C+Jo%CC%88rg+Sander%2C+Martin+Ester%2C+Hans+Peter+Kriegel%2C+and+Xiaowei+Xu%2E+2017%2E+Dbscan+revisited%2C+revisited%3A+why+and+how+you+should+%28still%29+use+dbscan%2E+ACM+Transactions+on+Database+Systems+%28TODS)
17. Hinrich Schütze. 1998. Automatic word sense discrimination. Computational linguistics, 24(1):97–123. [find](https://scholar.google.com/scholar?q=Hinrich+Schu%CC%88tze%2E+1998%2E+Automatic+word+sense+discrimination%2E+Computational+linguistics%2C+24%281%29%3A97%E2%80%93123%2E)
18. Goldee Udani, Shachi Dave, Anthony Davis, and Tim Sibley. 2005. Noun sense induction using web search results. In Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, pages 657– 658. [find](https://scholar.google.com/scholar?q=Goldee+Udani%2C+Shachi+Dave%2C+Anthony+Davis%2C+and+Tim+Sibley%2E+2005%2E+Noun+sense+induction+using+web+search+results%2E+In+Proceedings+of+the+28th+annual+international+ACM+SIGIR+conference+on+Research+and+deve)
19. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+neural+information+processi)