[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/aclbulk-2026-elboher-hebrew-diacritics-restoration-visual-representation/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [Ancient Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/hbo/) / Hebrew Diacritics Restoration using Visual Representation

# Hebrew Diacritics Restoration using Visual Representation

**Authors:** Yair Elboher; [Yuval Pinter](https://md.mova.elamur.ai/free/en/docs/papers/authors/yuval-pinter/) · **Year:** 2026 · **Venue:** Proceedings of the Second Workshop on Language Models for Low-Resource Languages (LoResLM 2026) **Link:** [aclanthology.org/2026.loreslm-1.44](https://aclanthology.org/2026.loreslm-1.44/) · library `aclbulk-2026-elboher-hebrew-diacritics-restoration-visual-representation` · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) (ACL Anthology)

[**0** citations](https://openalex.org/works?filter=cites:W7140141170) openalex · 2026-10-18[**1** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/aclbulk-2026-elboher-hebrew-diacritics-restoration-visual-representation/) of the 140673 papers whose bibliographies we read[Proceedings of the Second Workshop on Language Models for Low-Resource Languages (LoResLM 2026)](https://aclanthology.org/volumes/2026.loreslm-1/)[licence](https://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://aclanthology.org/2026.loreslm-1.44/)
- [PDF](https://aclanthology.org/2026.loreslm-1.44.pdf)
- [DOI 10.18653/v1/2026.loreslm-1.44](https://doi.org/10.18653/v1/2026.loreslm-1.44)
- [OpenAlex](https://openalex.org/W7140141170)

**Authors (2)**

1. [Yair Elboher](https://openalex.org/A5099978273) h 1 · 1 citations · 4 works
2. [Yuval Pinter](https://openalex.org/A5008477284) [ORCID](https://orcid.org/0000-0003-3174-1621) · h 14 · 1726 citations · 69 works

Citations and author metrics from openalex (2026-10-18); authors with the same name are not always told apart.

## What it is about

The system DiVRIT restores Hebrew niqqud at word level, as a candidate-selection task. A candidate generator (k-nearest neighbours over undiacritized forms of the same length, using the frequency-sorted pointing patterns of each neighbour) produces two or three pointed candidates per word. A visual encoder, a PIXEL-style vision transformer pretrained on rendered Hebrew, embeds each candidate as an image, and a text encoder (AlephBERT GIMMEL-small) embeds the context. The best candidate is picked by an inner product. Evaluated on a 20K-token Modern Hebrew test set, the model reaches 92.7% word accuracy in the oracle setting (the right form is always among the candidates) and 87.9% with its own generator. Nakdimon scores 89.8% and MenakBERT 94.1%, and the hybrid Dicta system 95.8%. Majority-class and one-neighbour baselines reach 84.9% and 87.1%. Auxiliary losses, balanced sampling and more candidates did not help; mirroring the candidate images helped at three candidates.

## Ideas

- I1. A rule-based candidate generator: take all pointed forms of each consonantal word from a lexicon with frequencies (for example from `macula dict`), filter them by the shape of the word, and rank them by frequency. Measure coverage (how often the right form is in the set) against candidate-set size, as the paper does; its curve suggests three to four candidates.
- I2. Report the no-learning floor on the same test set: the most-frequent-pointing baseline and a one-neighbour baseline. Any Mova ranking must beat both, with word accuracy and the separate decision accuracy for dagesh and shin dots.
- I3. The oracle-versus-generator gap (about 5 points here) is the cost of candidate generation, not of scoring. For Mova, the first target is coverage of the candidate set, and the ranker comes second.

## For Mova

I1 and I2 belong in the unpointed mode of `tools/hbo` and in [1. Script and pointing](https://md.mova.elamur.ai/free/en/docs/languages/hbo/01-script/) (what the niqqud encode, and which distinctions Modern and Biblical readings share). I3 is a plan for the order of work in `tools/hbo`. The paper is on Modern Hebrew, so its numbers are not a claim about Biblical or Rabbinic text.

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.18653/v1/2026.loreslm-1.44](https://doi.org/10.18653/v1/2026.loreslm-1.44)

```
@inproceedings{elboher2026hebrew,
  title = {{Hebrew Diacritics Restoration using Visual Representation}},
  author = {Elboher, Yair and Pinter, Yuval},
  booktitle = {Proceedings of the Second Workshop on Language Models for Low-Resource Languages (LoResLM 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.loreslm-1.44},
  url = {https://doi.org/10.18653/v1/2026.loreslm-1.44}
}
```

- [← previous A Layered Annotation Workflow for Semitic Epigraphy](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2026-bernstein-layered-annotation-workflow-semitic-epigraphy/)
- [next → Human-AI Annotation Error Auditing for Hebrew Diacritization with Frontier LLMs](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2026-gershuni-human-ai-annotation-error-auditing/)
- [↑ back to Ancient Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/hbo/)

## References

39 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yonatan Belinkov and James Glass. 2015. Arabic diacritization with recurrent neural networks. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2281– 2285, Lisbon, Portugal. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yonatan+Belinkov+and+James+Glass%2E+2015%2E+Arabic+diacritization+with+recurrent+neural+networks%2E+In+Proceedings+of+the+2015+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+2281%E2%80%93+228) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2015-belinkov-arabic-diacritization-recurrent-neural-networks/)
2. Nadav Borenstein, Phillip Rust, Desmond Elliott, and Isabelle Augenstein. 2023. PHD: Pixel-based language modeling of historical documents. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 87–107, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Nadav+Borenstein%2C+Phillip+Rust%2C+Desmond+Elliott%2C+and+Isabelle+Augenstein%2E+2023%2E+PHD%3A+Pixel%2Dbased+language+modeling+of+historical+documents%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Methods+i)
3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, and 12 others. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Tom+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+Sandhini+Agarwal%2C+Ariel+Herbert%2DVoss%2C+Gretchen)
4. Wei-Rui Chen, Ife Adebara, and Muhammad Abdul-Mageed. 2024. Interplay of machine translation, diacritics, and diacritization. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 7559–7601, Mexico City, Mexico. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Wei%2DRui+Chen%2C+Ife+Adebara%2C+and+Muhammad+Abdul%2DMageed%2E+2024%2E+Interplay+of+machine+translation%2C+diacritics%2C+and+diacritization%2E+In+Proceedings+of+the+2024+Conference+of+the+North+American+Chapter+of+the)
5. Ido Cohen, Jacob Gidron, and Idan Pinto. 2024. Menakbert – hebrew diacriticizer. arXiv preprint arXiv:2410.02417. [↗](https://arxiv.org/abs/2410.02417) [our card](https://md.mova.elamur.ai/free/en/docs/papers/hbo-2024-cohen-menakbert-hebrew-diacriticizer/)
6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
7. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An image is worth 16x16 words: Transformers for image recognition at scale. Preprint, arXiv:2010.11929. [↗](https://arxiv.org/abs/2010.11929)
8. Salman Elgamal, Ossama Obeid, Mhd Kabbani, Go Inoue, and Nizar Habash. 2024. Arabic diacritics in the Bar Gazit, Shaltiel Shmidman, Avi Shmidman, and Yuval Pinter. 2025. Splintering nonconcatenative languages for better tokenization. In Findings of the Association for Computational Linguistics: ACL 2025, pages 22405–22417, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Salman+Elgamal%2C+Ossama+Obeid%2C+Mhd+Kabbani%2C+Go+Inoue%2C+and+Nizar+Habash%2E+2024%2E+Arabic+diacritics+in+the+Bar+Gazit%2C+Shaltiel+Shmidman%2C+Avi+Shmidman%2C+and+Yuval+Pinter%2E+2025%2E+Splintering+nonconcatenative+l)
9. Elazar Gershuni and Yuval Pinter. 2022. Restoring Hebrew diacritics without a dictionary. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 1010–1018, Seattle, United States. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Elazar+Gershuni+and+Yuval+Pinter%2E+2022%2E+Restoring+Hebrew+diacritics+without+a+dictionary%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+NAACL+2022%2C+pages+1010%E2%80%931018%2C+Seattle%2C+United+Sta) [our card](https://md.mova.elamur.ai/free/en/docs/papers/findings-2022-gershuni-restoring-hebrew-diacritics-without-dictionary/)
10. Kyle Gorman and Yuval Pinter. 2025. Don‘t touch my diacritics. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pages 285–291, Albuquerque, New Mexico. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kyle+Gorman+and+Yuval+Pinter%2E+2025%2E+Don%E2%80%98t+touch+my+diacritics%2E+In+Proceedings+of+the+2025+Conference+of+the+Nations+of+the+Americas+Chapter+of+the+Association+for+Computational+Linguistics%3A+Human+Lang)
11. Eylon Gueta, Omer Goldman, and Reut Tsarfaty. 2023a. Explicit morphological knowledge improves pre-training of language models for hebrew. Preprint, arXiv:2311.00658. [↗](https://arxiv.org/abs/2311.00658) [our card](https://md.mova.elamur.ai/free/en/docs/papers/langstream-2023-gueta-explicit-morphological-knowledge-improves-pre/)
12. Eylon Gueta, Avi Shmidman, Shaltiel Shmidman, Cheyn Shmuel Shmidman, Joshua Guedalia, Moshe Koppel, Dan Bareket, Amit Seker, and Reut Tsarfaty. 2023b. Large pre-trained models with extra-large vocabularies: A contrastive analysis of hebrew bert models and a new one to outperform them all. Preprint, arXiv:2211.15199. [↗](https://arxiv.org/abs/2211.15199) [our card](https://md.mova.elamur.ai/free/en/docs/papers/hbo-2022-gueta-large-pre-trained-models-extra/)
13. Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. 2022. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16000–16009. [find](https://scholar.google.com/scholar?q=Kaiming+He%2C+Xinlei+Chen%2C+Saining+Xie%2C+Yanghao+Li%2C+Piotr+Doll%C3%A1r%2C+and+Ross+Girshick%2E+2022%2E+Masked+autoencoders+are+scalable+vision+learners%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+)
14. Edward Klyshinsky, Olga Karpik, and Alexander Bondarenko. 2021. A comparison of neural networks architectures for diacritics restoration. In Recent Trends in Analysis of Images, Social Networks and Texts, pages 235–245. Springer. [find](https://scholar.google.com/scholar?q=Edward+Klyshinsky%2C+Olga+Karpik%2C+and+Alexander+Bondarenko%2E+2021%2E+A+comparison+of+neural+networks+architectures+for+diacritics+restoration%2E+In+Recent+Trends+in+Analysis+of+Images%2C+Social+Networks+and+Te)
15. Vladimir I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady, 10(8):707–710. [find](https://scholar.google.com/scholar?q=Vladimir+I%2E+Levenshtein%2E+1966%2E+Binary+codes+capable+of+correcting+deletions%2C+insertions%2C+and+reversals%2E+Soviet+Physics+Doklady%2C+10%288%29%3A707%E2%80%93710%2E)
16. Omer Levy, Yoav Goldberg, and Ido Dagan. 2015. Improving distributional similarity with lessons learned from word embeddings. Transactions of the Association for Computational Linguistics, 3:211–225. [find](https://scholar.google.com/scholar?q=Omer+Levy%2C+Yoav+Goldberg%2C+and+Ido+Dagan%2E+2015%2E+Improving+distributional+similarity+with+lessons+learned+from+word+embeddings%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+3%3A211%E2%80%93225%2E)
17. Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. 2021. Align before fuse: Vision and language representation learning with momentum distillation. In Advances in Neural Information Processing Systems, volume 34, pages 9694–9705. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Junnan+Li%2C+Ramprasaath+Selvaraju%2C+Akhilesh+Gotmare%2C+Shafiq+Joty%2C+Caiming+Xiong%2C+and+Steven+Chu+Hong+Hoi%2E+2021%2E+Align+before+fuse%3A+Vision+and+language+representation+learning+with+momentum+distillation)
18. Jonas Lotz, Elizabeth Salesky, Phillip Rust, and Desmond Elliott. 2023. Text rendering strategies for pixel language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10155–10172, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jonas+Lotz%2C+Elizabeth+Salesky%2C+Phillip+Rust%2C+and+Desmond+Elliott%2E+2023%2E+Text+rendering+strategies+for+pixel+language+models%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Methods+in+Natural+Langu)
19. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Ilya+Sutskever%2C+Kai+Chen%2C+Greg+S+Corrado%2C+and+Jeff+Dean%2E+2013%2E+Distributed+representations+of+words+and+phrases+and+their+compositionality%2E+In+Advances+in+Neural+Information+Processing+)
20. Alberto Muñoz-Ortiz, Verena Blaschke, and Barbara Plank. 2025. Evaluating pixel language models on non-standardized languages. In Proceedings of the 31st International Conference on Computational Linguistics, pages 6412–6419, Abu Dhabi, UAE. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Alberto+Mu%C3%B1oz%2DOrtiz%2C+Verena+Blaschke%2C+and+Barbara+Plank%2E+2025%2E+Evaluating+pixel+language+models+on+non%2Dstandardized+languages%2E+In+Proceedings+of+the+31st+International+Conference+on+Computational+Ling)
21. Jakub Náplava, Milan Straka, and Jana Straková. 2021. Diacritics restoration using bert with analysis on czech language. Prague Bulletin of Mathematical Linguistics, 116(1):27–42. [find](https://scholar.google.com/scholar?q=Jakub+N%C3%A1plava%2C+Milan+Straka%2C+and+Jana+Strakov%C3%A1%2E+2021%2E+Diacritics+restoration+using+bert+with+analysis+on+czech+language%2E+Prague+Bulletin+of+Mathematical+Linguistics%2C+116%281%29%3A27%E2%80%9342%2E)
22. Pedro Javier Ortiz Suárez, Benoit Sagot, and Laurent Romary. 2019. Asynchronous pipelines for processing huge corpora on medium to low resource infrastructures. In Proceedings of the Workshop on Challenges in the Management of Large Corpora (CMLC-7), pages 9–16, Cardiff, UK. Leibniz-Institut für Deutsche Sprache. [find](https://scholar.google.com/scholar?q=Pedro+Javier+Ortiz+Su%C3%A1rez%2C+Benoit+Sagot%2C+and+Laurent+Romary%2E+2019%2E+Asynchronous+pipelines+for+processing+huge+corpora+on+medium+to+low+resource+infrastructures%2E+In+Proceedings+of+the+Workshop+on+Chall)
23. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 8748–8763. PMLR. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jong+Wook+Kim%2C+Chris+Hallacy%2C+Aditya+Ramesh%2C+Gabriel+Goh%2C+Sandhini+Agarwal%2C+Girish+Sastry%2C+Amanda+Askell%2C+Pamela+Mishkin%2C+Jack+Clark%2C+Gretchen+Krueger%2C+and+Ilya+Sutskever%2E+2021%2E+Learning)
24. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, Hong Kong, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Nils+Reimers+and+Iryna+Gurevych%2E+2019%2E+Sentence%2DBERT%3A+Sentence+embeddings+using+Siamese+BERT%2Dnetworks%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+the)
25. Qiang Ren and Junli Wang. 2025. Irrelevant patch-masked autoencoders for enhancing vision transformers under limited data. Knowledge-Based Systems, 310:112936. [find](https://scholar.google.com/scholar?q=Qiang+Ren+and+Junli+Wang%2E+2025%2E+Irrelevant+patch%2Dmasked+autoencoders+for+enhancing+vision+transformers+under+limited+data%2E+Knowledge%2DBased+Systems%2C+310%3A112936%2E)
26. Phillip Rust, Jonas F. Lotz, Emanuele Bugliarello, Elizabeth Salesky, Miryam de Lhoneux, and Desmond Elliott. 2023. Language modelling with pixels. In The Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Phillip+Rust%2C+Jonas+F%2E+Lotz%2C+Emanuele+Bugliarello%2C+Elizabeth+Salesky%2C+Miryam+de+Lhoneux%2C+and+Desmond+Elliott%2E+2023%2E+Language+modelling+with+pixels%2E+In+The+Eleventh+International+Conference+on+Learning)
27. Elizabeth Salesky, David Etter, and Matt Post. 2021. Robust open-vocabulary translation from visual text representations. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7235–7252, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Elizabeth+Salesky%2C+David+Etter%2C+and+Matt+Post%2E+2021%2E+Robust+open%2Dvocabulary+translation+from+visual+text+representations%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language)
28. Sara Shatnawi, Sawsan Alqahtani, and Hanan Aldarmaki. 2024. Automatic restoration of diacritics for speech data sets. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 4166–4176, Mexico City, Mexico. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sara+Shatnawi%2C+Sawsan+Alqahtani%2C+and+Hanan+Aldarmaki%2E+2024%2E+Automatic+restoration+of+diacritics+for+speech+data+sets%2E+In+Proceedings+of+the+2024+Conference+of+the+North+American+Chapter+of+the+Associa)
29. Avi Shmidman, Shaltiel Shmidman, Moshe Koppel, and Yoav Goldberg. 2020. Nakdan: Professional Hebrew diacritizer. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 197–203, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Avi+Shmidman%2C+Shaltiel+Shmidman%2C+Moshe+Koppel%2C+and+Yoav+Goldberg%2E+2020%2E+Nakdan%3A+Professional+Hebrew+diacritizer%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Computational+Linguist) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2020-shmidman-nakdan-professional-hebrew-diacritizer/)
30. Leon Sick, Dominik Engel, Pedro Hermosilla, and Timo Ropinski. 2025. Attention-guided masked autoencoders for learning image representations. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 836–846. [find](https://scholar.google.com/scholar?q=Leon+Sick%2C+Dominik+Engel%2C+Pedro+Hermosilla%2C+and+Timo+Ropinski%2E+2025%2E+Attention%2Dguided+masked+autoencoders+for+learning+image+representations%2E+In+2025+IEEE%2FCVF+Winter+Conference+on+Applications+of+Comp)
31. Richard Socher, Milind Ganjoo, Christopher D Manning, and Andrew Ng. 2013. Zero-shot learning through cross-modal transfer. In Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Milind+Ganjoo%2C+Christopher+D+Manning%2C+and+Andrew+Ng%2E+2013%2E+Zero%2Dshot+learning+through+cross%2Dmodal+transfer%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+volume+26%2E+Curran+Asso)
32. Yintao Tai, Xiyang Liao, Alessandro Suglia, and Antonio Vergari. 2024. PIXAR: Auto-regressive language modeling in pixel space. In Findings of the Association for Computational Linguistics: ACL 2024, pages 14673–14695, Bangkok, Thailand. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yintao+Tai%2C+Xiyang+Liao%2C+Alessandro+Suglia%2C+and+Antonio+Vergari%2E+2024%2E+PIXAR%3A+Auto%2Dregressive+language+modeling+in+pixel+space%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2024%2C+)
33. Reut Tsarfaty, Shoval Sadde, Stav Klein, and Amit Seker. 2019. What‘s wrong with Hebrew NLP? and how to make it right. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations, pages 259–264, Hong Kong, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Reut+Tsarfaty%2C+Shoval+Sadde%2C+Stav+Klein%2C+and+Amit+Seker%2E+2019%2E+What%E2%80%98s+wrong+with+Hebrew+NLP%3F+and+how+to+make+it+right%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Pr) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2019-tsarfaty-what-s-wrong-hebrew-nlp/)
34. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Us-zoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Us%2Dzoreit%2C+Llion+Jones%2C+Aidan+N%2E+Gomez%2C+Lukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+Neural+Information+Process)
35. Yongqin Xian, Christoph H. Lampert, Bernt Schiele, and Zeynep Akata. 2019. Zero-shot learning—a comprehensive evaluation of the good, the bad and the ugly. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(9):2251–2265. [find](https://scholar.google.com/scholar?q=Yongqin+Xian%2C+Christoph+H%2E+Lampert%2C+Bernt+Schiele%2C+and+Zeynep+Akata%2E+2019%2E+Zero%2Dshot+learning%E2%80%94a+comprehensive+evaluation+of+the+good%2C+the+bad+and+the+ugly%2E+IEEE+Transactions+on+Pattern+Analysis+and+Ma)
36. Qinyuan Ye, Belinda Z. Li, Sinong Wang, Benjamin Bolte, Hao Ma, Wen-tau Yih, Xiang Ren, and Madian Khabsa. 2021. On the influence of masking policies in intermediate pre-training. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7190–7202, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Qinyuan+Ye%2C+Belinda+Z%2E+Li%2C+Sinong+Wang%2C+Benjamin+Bolte%2C+Hao+Ma%2C+Wen%2Dtau+Yih%2C+Xiang+Ren%2C+and+Madian+Khabsa%2E+2021%2E+On+the+influence+of+masking+policies+in+intermediate+pre%2Dtraining%2E+In+Proceedings+of+th)
37. Wenpeng Yin, Jamaal Hay, and Dan Roth. 2019. Benchmarking zero-shot text classification: Datasets, evaluation and entailment approach. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3914–3923, Hong Kong, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Wenpeng+Yin%2C+Jamaal+Hay%2C+and+Dan+Roth%2E+2019%2E+Benchmarking+zero%2Dshot+text+classification%3A+Datasets%2C+evaluation+and+entailment+approach%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Nat)
38. Li Yujian and Liu Bo. 2007. A normalized levenshtein distance metric. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(6):1091–1095. [find](https://scholar.google.com/scholar?q=Li+Yujian+and+Liu+Bo%2E+2007%2E+A+normalized+levenshtein+distance+metric%2E+IEEE+Transactions+on+Pattern+Analysis+and+Machine+Intelligence%2C+29%286%29%3A1091%E2%80%931095%2E)
39. Lei Zhou, Huidong Liu, Joseph Bae, Junjun He, Dimitris Samaras, and Prateek Prasanna. 2023. Self pre-training with masked autoencoders for medical image classification and segmentation. In 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), pages 1–6. [find](https://scholar.google.com/scholar?q=Lei+Zhou%2C+Huidong+Liu%2C+Joseph+Bae%2C+Junjun+He%2C+Dimitris+Samaras%2C+and+Prateek+Prasanna%2E+2023%2E+Self+pre%2Dtraining+with+masked+autoencoders+for+medical+image+classification+and+segmentation%2E+In+2023+IEEE+2)