[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/langstream-2023-gueta-explicit-morphological-knowledge-improves-pre/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [Modern Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/he/) / Explicit Morphological Knowledge Improves Pre-training of Language Models for Hebrew

# Explicit Morphological Knowledge Improves Pre-training of Language Models for Hebrew

**Authors:** [Eylon Gueta](https://md.mova.elamur.ai/free/en/docs/papers/authors/eylon-gueta/); [Omer Goldman](https://md.mova.elamur.ai/free/en/docs/papers/authors/omer-goldman/); [Reut Tsarfaty](https://md.mova.elamur.ai/free/en/docs/papers/authors/reut-tsarfaty/) · **Year:** 2023 · **Venue:** arXiv:2311.00658 **Link:** [arxiv.org/abs/2311.00658](https://arxiv.org/abs/2311.00658) · library `langstream-2023-gueta-explicit-morphological-knowledge-improves-pre` · **Licence:** [CC BY-NC-ND 4.0](https://creativecommons.org/licenses/by-nc-nd/4.0/) (arXiv) — summary only

[**0** citations](https://openalex.org/works?filter=cites:W4388276214) openalex · 2026-10-18[**7** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/langstream-2023-gueta-explicit-morphological-knowledge-improves-pre/) of the 140673 papers whose bibliographies we read[arXiv:2311.00658](https://openalex.org/S4306400194)[licence](http://creativecommons.org/licenses/by-nc-nd/4.0/)

- [Publisher's page](https://arxiv.org/abs/2311.00658)
- [PDF](https://arxiv.org/pdf/2311.00658)
- [arXiv 2311.00658](https://arxiv.org/abs/2311.00658)
- [OpenAlex](https://openalex.org/W4388276214)

**Authors (3)**

1. Eylon Gueta
2. [Omer Goldman](https://openalex.org/A5028852856) [ORCID](https://orcid.org/0000-0001-5863-4856) · h 12 · 868 citations · 136 works
3. [Reut Tsarfaty](https://openalex.org/A5063283689) [ORCID](https://orcid.org/0000-0001-6406-6568) · h 24 · 2886 citations · 217 works

Citations and author metrics from openalex (2026-10-18); authors with the same name are not always told apart.

## What it is about

The question is whether a BERT-style Hebrew model learns better when its tokenizer is told about Hebrew morphology before pre-training, instead of a purely statistical WordPiece split. The authors compare three tokenizations, each at 16K, 32K and 64K vocabulary: plain WordPiece (baseline); morphological segmentation, where a trained segmenter splits each word into prefixes, host and suffix before WordPiece; and prefix-suffix separation, where a regular expression splits any valid prefix or suffix string without context. Models are pre-trained on Hebrew Wikipedia and a Hebrew web corpus, then evaluated on a benchmark of NER (word and morpheme level), question answering, homograph disambiguation, segmentation, POS, morphological features and dependency parsing. Morphological segmentation gives up to 3 F1 points on NER and about 1 F1 point on dependency parsing. The regular-expression split helps NER but hurts most other tasks, because it adds ambiguity. The larger vocabulary helps most on NER and QA.

## Ideas

- I1. Compare Mova's deterministic segmenter tables (the `pre` and `lex` splits in `tools/hbo`) against a plain regular-expression split on the same gold words. The paper shows that a context-free split adds ambiguity, so the context-sensitive table should win on the morpheme-level measure; if it does not, the table is wrong.
- I2. Check where NER errors come from: the paper's gain on NER comes from splitting the clitic prefixes (and, that, from, to) off named entities. A Mova test can count NER boundary errors where a clitic is glued to the name, and report them separately.
- I3. Do not chase the vocabulary-size effect: it is a property of subword models. Mova has no subword vocabulary to size; the lesson that more splits raise the subword count and can hurt is a reason to keep the segment inventory small.

## For Mova

I1 goes into `tools/hbo` (the segmenter tables) and is measured on the MACULA gold segmentation. I2 is a test for `tools/hbo` and for the Hebrew NER plan in [Pronouns, particles and prepositions](https://md.mova.elamur.ai/free/en/docs/languages/hbo/05-pronouns-particles/). I3 is a design note for [13. Biblical Hebrew in Mova: the tools](https://md.mova.elamur.ai/free/en/docs/languages/hbo/13-mova-tools/).

Topics: [Named entity recognition](https://md.mova.elamur.ai/free/en/docs/papers/topics/named-entity-recognition/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [arXiv 2311.00658](https://arxiv.org/abs/2311.00658)

```
@misc{gueta2023explicit,
  title = {{Explicit Morphological Knowledge Improves Pre-training of Language Models for Hebrew}},
  author = {Gueta, Eylon and Goldman, Omer and Tsarfaty, Reut},
  year = {2023},
  eprint = {2311.00658},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2311.00658}
}
```

- [← previous Style transfer and classification in hebrew news items](https://md.mova.elamur.ai/free/en/docs/papers/langstream-2022-weingarten-style-transfer-classification-hebrew-news/)
- [next → Offensive Hebrew Corpus and Detection using BERT](https://md.mova.elamur.ai/free/en/docs/papers/langstream-2023-hamad-offensive-hebrew-corpus-detection-using/)
- [↑ back to Modern Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/he/)

## References

44 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mohamed Alkaoud and Mairaj Syed. 2020. On the importance of tokenization in Arabic embedding models. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 119–129, Barcelona, Spain (Online). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mohamed+Alkaoud+and+Mairaj+Syed%2E+2020%2E+On+the+importance+of+tokenization+in+Arabic+embedding+models%2E+In+Proceedings+of+the+Fifth+Arabic+Natural+Language+Processing+Workshop%2C+pages+119%E2%80%93129%2C+Barcelona%2C+) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2020-alkaoud-importance-tokenization-arabic-embedding-models/)
2. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. [find](https://scholar.google.com/scholar?q=Wissam+Antoun%2C+Fady+Baly%2C+and+Hazem+Hajj%2E+2020%2E+AraBERT%3A+Transformer%2Dbased+model+for+Arabic+language+understanding%2E+In+Proceedings+of+the+4th+Workshop+on+Open%2DSource+Arabic+Corpora+and+Processing+Tool)
3. Ali Araabi, Christof Monz, and Vlad Niculae. 2022. How effective is byte pair encoding for out-of-vocabulary words in neural machine translation? In Proceedings of the 15th biennial conference of the Association for Machine Translation in the Americas (Volume 1: Research Track), pages 117–130, Orlando, USA. Association for Machine Translation in the Americas. [find](https://scholar.google.com/scholar?q=Ali+Araabi%2C+Christof+Monz%2C+and+Vlad+Niculae%2E+2022%2E+How+effective+is+byte+pair+encoding+for+out%2Dof%2Dvocabulary+words+in+neural+machine+translation%3F+In+Proceedings+of+the+15th+biennial+conference+of+the+)
4. Giuseppe Attardi, Daniele Sartiano, and Maria Simi. 2021. Biaffine dependency and semantic graph parsing for EnhancedUniversal dependencies. In Proceedings of the 17th International Conference on Parsing Technologies and the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies (IWPT 2021), pages 184–188, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Giuseppe+Attardi%2C+Daniele+Sartiano%2C+and+Maria+Simi%2E+2021%2E+Biaffine+dependency+and+semantic+graph+parsing+for+EnhancedUniversal+dependencies%2E+In+Proceedings+of+the+17th+International+Conference+on+Pars)
5. Oded Avraham and Yoav Goldberg. 2017. The interplay of semantics and morphology in word embeddings. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 422–426, Valencia, Spain. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Oded+Avraham+and+Yoav+Goldberg%2E+2017%2E+The+interplay+of+semantics+and+morphology+in+word+embeddings%2E+In+Proceedings+of+the+15th+Conference+of+the+European+Chapter+of+the+Association+for+Computational+L)
6. Dan Bareket and Reut Tsarfaty. 2021. Neural modeling for named entities and morphology (NEMO2). Transactions of the Association for Computational Linguistics, 9:909–928. [find](https://scholar.google.com/scholar?q=Dan+Bareket+and+Reut+Tsarfaty%2E+2021%2E+Neural+modeling+for+named+entities+and+morphology+%28NEMO2%29%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+9%3A909%E2%80%93928%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/tacl-2021-bareket-neural-modeling-named-entities-morphology/)
7. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146. [↗](https://doi.org/10.1162/tacl_a_00051)
8. Idan Brusilovsky and Reut Tsarfaty. 2022. Neural token segmentation for high token-internal complexity. arXiv preprint arXiv:2203.10845. [↗](https://arxiv.org/abs/2203.10845)
9. Kris Cao and Laura Rimell. 2021. You should evaluate your language model on marginal likelihood over tokenisations. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2104–2114, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kris+Cao+and+Laura+Rimell%2E+2021%2E+You+should+evaluate+your+language+model+on+marginal+likelihood+over+tokenisations%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Proce)
10. Avihay Chriqui and Inbal Yahav. 2022. Hebert and hebemo: A hebrew bert model and a tool for polarity analysis and emotion recognition. INFORMS Journal on Data Science, 1(1):81–95. [↗](https://doi.org/10.1287/ijds.2022.0016) [our card](https://md.mova.elamur.ai/free/en/docs/papers/langstream-2021-chriqui-hebert-hebemo-hebrew-bert-model/)
11. Jonathan H. Clark, Dan Garrette, Iulia Turc, and John Wieting. 2022. Canine: Pre-training an efficient tokenization-free encoder for language representation. Transactions of the Association for Computational Linguistics, 10:73–91. Amir DN Cohen, Hilla Merhav Fine, Yoav Goldberg, and Reut Tsarfaty. Heq: a large and diverse hebrew reading comprehension benchmark. [find](https://scholar.google.com/scholar?q=Jonathan+H%2E+Clark%2C+Dan+Garrette%2C+Iulia+Turc%2C+and+John+Wieting%2E+2022%2E+Canine%3A+Pre%2Dtraining+an+efficient+tokenization%2Dfree+encoder+for+language+representation%2E+Transactions+of+the+Association+for+Comput) [our card](https://md.mova.elamur.ai/free/en/docs/papers/findings-2023-cohen-heq-large-diverse-hebrew-reading/)
12. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440– 8451, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2020.acl-main.747)
13. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
14. Zhangyin Feng, Duyu Tang, Cong Zhou, Junwei Liao, Shuangzhi Wu, Xiaocheng Feng, Bing Qin, Yunbo Cao, and Shuming Shi. 2022. Pretraining without wordpieces: learning over a vocabulary of millions of words. arXiv preprint arXiv:2202.12142. [↗](https://arxiv.org/abs/2202.12142)
15. Eylon Guetta, Avi Shmidman, Shaltiel Shmidman, Cheyn Shmuel Shmidman, Joshua Guedalia, Moshe Koppel, Dan Bareket, Amit Seker, and Reut Tsarfaty. 2022. Large pre-trained models with extra-large vocabularies: A contrastive analysis of hebrew bert models and a new one to outperform them all. arXiv preprint arXiv:2211.15199. [↗](https://arxiv.org/abs/2211.15199) [our card](https://md.mova.elamur.ai/free/en/docs/papers/hbo-2022-gueta-large-pre-trained-models-extra/)
16. Valentin Hofmann, Janet Pierrehumbert, and Hinrich Schütze. 2021. Superbizarre is not superb: Derivational morphology improves BERT’s interpretation of complex words. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3594–3608, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Valentin+Hofmann%2C+Janet+Pierrehumbert%2C+and+Hinrich+Sch%C3%BCtze%2E+2021%2E+Superbizarre+is+not+superb%3A+Derivational+morphology+improves+BERT%E2%80%99s+interpretation+of+complex+words%2E+In+Proceedings+of+the+59th+Annual) [our card](https://md.mova.elamur.ai/free/en/docs/papers/lex-2021-hofmann-superbizarre-derivational/)
17. Peter Izsak, Moshe Berchansky, and Omer Levy. 2021. How to train BERT with an academic budget. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 10644– 10652, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Peter+Izsak%2C+Moshe+Berchansky%2C+and+Omer+Levy%2E+2021%2E+How+to+train+BERT+with+an+academic+budget%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+10644%E2%80%93+1)
18. Omri Keren, Tal Avinari, Reut Tsarfaty, and Omer Levy. 2022. Breaking character: Are subwords good enough for mrls after all? arXiv preprint arXiv:2204.04748. [↗](https://arxiv.org/abs/2204.04748)
19. Omri Keren and Omer Levy. 2021. ParaShoot: A Hebrew question answering dataset. In Proceedings of the 3rd Workshop on Machine Reading for Question Answering, pages 106–112, Punta Cana, Dominican Republic. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Omri+Keren+and+Omer+Levy%2E+2021%2E+ParaShoot%3A+A+Hebrew+question+answering+dataset%2E+In+Proceedings+of+the+3rd+Workshop+on+Machine+Reading+for+Question+Answering%2C+pages+106%E2%80%93112%2C+Punta+Cana%2C+Dominican+Repub) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2021-keren-parashoot-hebrew-question-answering-dataset/)
20. Stav Klein and Reut Tsarfaty. 2020. Getting the [find](https://scholar.google.com/scholar?q=Stav+Klein+and+Reut+Tsarfaty%2E+2020%2E+Getting+the)
21. Manuel Mager, Arturo Oncevay, Elisabeth Mager, Katharina Kann, and Thang Vu. 2022. BPE vs. morphological segmentation: A case study on machine translation of four polysynthetic languages. In Findings of the Association for Computational Linguistics: ACL 2022, pages 961–971, Dublin, Ireland. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Manuel+Mager%2C+Arturo+Oncevay%2C+Elisabeth+Mager%2C+Katharina+Kann%2C+and+Thang+Vu%2E+2022%2E+BPE+vs%2E+morphological+segmentation%3A+A+case+study+on+machine+translation+of+four+polysynthetic+languages%2E+In+Findings+)
22. John J McCarthy. 1981. A prosodic theory of nonconcatenative morphology. Linguistic inquiry, 12(3):373–418. [find](https://scholar.google.com/scholar?q=John+J+McCarthy%2E+1981%2E+A+prosodic+theory+of+nonconcatenative+morphology%2E+Linguistic+inquiry%2C+12%283%29%3A373%E2%80%93418%2E)
23. Anssi Moisio, Mathias Creutz, and Mikko Kurimo. 2023. Evaluating morphological generalisation in machine translation by distribution-based compositionality assessment. In Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa), pages 738–751, Tórshavn, Faroe Islands. University of Tartu Library. [find](https://scholar.google.com/scholar?q=Anssi+Moisio%2C+Mathias+Creutz%2C+and+Mikko+Kurimo%2E+2023%2E+Evaluating+morphological+generalisation+in+machine+translation+by+distribution%2Dbased+compositionality+assessment%2E+In+Proceedings+of+the+24th+Nordi)
24. Naama Ben Mordecai and Michael Elhadad. 2005. Hebrew named entity recognition. MONEY, 81(83.93):82–49. [find](https://scholar.google.com/scholar?q=Naama+Ben+Mordecai+and+Michael+Elhadad%2E+2005%2E+Hebrew+named+entity+recognition%2E+MONEY%2C+81%2883%2E93%29%3A82%E2%80%9349%2E)
25. Minh Van Nguyen, Viet Dac Lai, Amir Pouran Ben Veyseh, and Thien Huu Nguyen. 2021. Trankit: A lightweight transformer-based toolkit for multilingual natural language processing. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 80–90, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Minh+Van+Nguyen%2C+Viet+Dac+Lai%2C+Amir+Pouran+Ben+Veyseh%2C+and+Thien+Huu+Nguyen%2E+2021%2E+Trankit%3A+A+lightweight+transformer%2Dbased+toolkit+for+multilingual+natural+language+processing%2E+In+Proceedings+of+the+)
26. Antoine Nzeyimana and Andre Niyongabo Rubungo. 2022. KinyaBERT: a morphology-aware Kinyarwanda language model. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5347–5363, Dublin, Ireland. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Antoine+Nzeyimana+and+Andre+Niyongabo+Rubungo%2E+2022%2E+KinyaBERT%3A+a+morphology%2Daware+Kinyarwanda+language+model%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+the+Association+for+Computational+Linguistic)
27. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12:2825–2830. [find](https://scholar.google.com/scholar?q=Fabian+Pedregosa%2C+Ga%C3%ABl+Varoquaux%2C+Alexandre+Gramfort%2C+Vincent+Michel%2C+Bertrand+Thirion%2C+Olivier+Grisel%2C+Mathieu+Blondel%2C+Peter+Prettenhofer%2C+Ron+Weiss%2C+Vincent+Dubourg%2C+et+al%2E+2011%2E+Scikit%2Dlearn%3A+Mach)
28. Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018. Improving language understanding by generative pre-training. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Karthik+Narasimhan%2C+Tim+Salimans%2C+Ilya+Sutskever%2C+et+al%2E+2018%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E)
29. Shoval Sade, Amit Seker, and Reut Tsarfaty. 2018. The Hebrew Universal Dependency treebank: Past present and future. In Proceedings of the Second Workshop on Universal Dependencies (UDW 2018), pages 133–143, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Shoval+Sade%2C+Amit+Seker%2C+and+Reut+Tsarfaty%2E+2018%2E+The+Hebrew+Universal+Dependency+treebank%3A+Past+present+and+future%2E+In+Proceedings+of+the+Second+Workshop+on+Universal+Dependencies+%28UDW+2018%29%2C+pages+1) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2018-sade-hebrew-universal-dependency-treebank-past/)
30. Timo Schick and Hinrich Schütze. 2020. Rare words: A major problem for contextualized embeddings and how to fix it by attentive mimicking. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8766–8774. [find](https://scholar.google.com/scholar?q=Timo+Schick+and+Hinrich+Sch%C3%BCtze%2E+2020%2E+Rare+words%3A+A+major+problem+for+contextualized+embeddings+and+how+to+fix+it+by+attentive+mimicking%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intellige)
31. Mike Schuster and Kaisuke Nakajima. 2012. Japanese and korean voice search. In 2012 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 5149–5152. IEEE. [find](https://scholar.google.com/scholar?q=Mike+Schuster+and+Kaisuke+Nakajima%2E+2012%2E+Japanese+and+korean+voice+search%2E+In+2012+IEEE+international+conference+on+acoustics%2C+speech+and+signal+processing+%28ICASSP%29%2C+pages+5149%E2%80%935152%2E+IEEE%2E)
32. Amit Seker, Elron Bandel, Dan Bareket, Idan Brusilovsky, Refael Greenfeld, and Reut Tsarfaty. 2022. AlephBERT: Language model pre-training and evaluation from sub-word to sentence level. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 46–56, Dublin, Ireland. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Amit+Seker%2C+Elron+Bandel%2C+Dan+Bareket%2C+Idan+Brusilovsky%2C+Refael+Greenfeld%2C+and+Reut+Tsarfaty%2E+2022%2E+AlephBERT%3A+Language+model+pre%2Dtraining+and+evaluation+from+sub%2Dword+to+sentence+level%2E+In+Proceeding) [our card](https://md.mova.elamur.ai/free/en/docs/papers/acl-2022-seker-alephbert-language-model-pre-training/)
33. Amit Seker, Amir More, and Reut Tsarfaty. 2018. Universal morpho-syntactic parsing and the contribution of lexica: Analyzing the ONLP lab submission to the CoNLL 2018 shared task. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 208–215, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Amit+Seker%2C+Amir+More%2C+and+Reut+Tsarfaty%2E+2018%2E+Universal+morpho%2Dsyntactic+parsing+and+the+contribution+of+lexica%3A+Analyzing+the+ONLP+lab+submission+to+the+CoNLL+2018+shared+task%2E+In+Proceedings+of+th) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2018-seker-universal-morpho-syntactic-parsing-contribution/)
34. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, Berlin, Germany. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016%2E+Neural+machine+translation+of+rare+words+with+subword+units%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Ling)
35. Vitaly Shalumov and Harel Haskey. 2023. Hero: Roberta and longformer hebrew language models. arXiv preprint arXiv:2304.11077. [↗](https://arxiv.org/abs/2304.11077) [our card](https://md.mova.elamur.ai/free/en/docs/papers/langstream-2023-shalumov-hero-roberta-longformer-hebrew-language/)
36. Avi Shmidman, Cheyn Shmuel Shmidman, Dan Bareket, Moshe Koppel, and Reut Tsarfaty. 2023. Do pre-trained contextual language models distinguish between Hebrew homograph analyses? In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 849–864, Dubrovnik, Croatia. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Avi+Shmidman%2C+Cheyn+Shmuel+Shmidman%2C+Dan+Bareket%2C+Moshe+Koppel%2C+and+Reut+Tsarfaty%2E+2023%2E+Do+pre%2Dtrained+contextual+language+models+distinguish+between+Hebrew+homograph+analyses%3F+In+Proceedings+of+the+)
37. Yi Tay, Vinh Q Tran, Sebastian Ruder, Jai Gupta, Hyung Won Chung, Dara Bahri, Zhen Qin, Simon Baumgartner, Cong Yu, and Donald Metzler. 2021. Charformer: Fast character transformers via gradient-based subword tokenization. arXiv preprint arXiv:2106.12672. [↗](https://arxiv.org/abs/2106.12672)
38. Reut Tsarfaty, Dan Bareket, Stav Klein, and Amit Seker. 2020. From SPMRL to NMRL: What did we learn (and unlearn) in a decade of parsing morphologically-rich languages (MRLs)? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7396–7408, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Reut+Tsarfaty%2C+Dan+Bareket%2C+Stav+Klein%2C+and+Amit+Seker%2E+2020%2E+From+SPMRL+to+NMRL%3A+What+did+we+learn+%28and+unlearn%29+in+a+decade+of+parsing+morphologically%2Drich+languages+%28MRLs%29%3F+In+Proceedings+of+the+58)
39. Reut Tsarfaty, Shoval Sadde, Stav Klein, and Amit Seker. 2019. What’s wrong with Hebrew NLP? and how to make it right. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations, pages 259–264, Hong Kong, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Reut+Tsarfaty%2C+Shoval+Sadde%2C+Stav+Klein%2C+and+Amit+Seker%2E+2019%2E+What%E2%80%99s+wrong+with+Hebrew+NLP%3F+and+how+to+make+it+right%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Pr) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2019-tsarfaty-what-s-wrong-hebrew-nlp/)
40. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Thomas+Wolf%2C+Lysandre+Debut%2C+Victor+Sanh%2C+Julien+Chaumond%2C+Clement+Delangue%2C+Anthony+Moi%2C+Pier%2Dric+Cistac%2C+Tim+Rault%2C+Remi+Louf%2C+Morgan+Funtowicz%2C+Joe+Davison%2C+Sam+Shleifer%2C+Patrick+von+Platen%2C+Clara+)
41. Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. 2022. ByT5: Towards a token-free future with pre-trained byte-to-byte models. Transactions of the Association for Computational Linguistics, 10:291–306. [↗](https://doi.org/10.1162/tacl_a_00461)
42. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mT5: A massively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 483–498, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Linting+Xue%2C+Noah+Constant%2C+Adam+Roberts%2C+Mihir+Kale%2C+Rami+Al%2DRfou%2C+Aditya+Siddhant%2C+Aditya+Barua%2C+and+Colin+Raffel%2E+2021%2E+mT5%3A+A+massively+multilingual+pre%2Dtrained+text%2Dto%2Dtext+transformer%2E+In+Procee)
43. Amir Zeldes. 2018. A characterwise windowed approach to Hebrew morphological segmentation. In Proceedings of the Fifteenth Workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 101–110, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Amir+Zeldes%2E+2018%2E+A+characterwise+windowed+approach+to+Hebrew+morphological+segmentation%2E+In+Proceedings+of+the+Fifteenth+Workshop+on+Computational+Research+in+Phonetics%2C+Phonology%2C+and+Morphology%2C+p) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2018-zeldes-characterwise-windowed-approach-hebrew-morphological/)
44. Amir Zeldes, Nick Howell, Noam Ordan, and Yifat Ben Moshe. 2022. A second wave of UD Hebrew treebanking and cross-domain parsing. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4331–4344, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Amir+Zeldes%2C+Nick+Howell%2C+Noam+Ordan%2C+and+Yifat+Ben+Moshe%2E+2022%2E+A+second+wave+of+UD+Hebrew+treebanking+and+cross%2Ddomain+parsing%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods+in+Natural+) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2022-zeldes-second-wave-ud-hebrew-treebanking/)