[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/findings-2023-eyal-multilingual-sequence-sequence-models-hebrew/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [Modern Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/he/) / Multilingual Sequence-to-Sequence Models for Hebrew NLP

# Multilingual Sequence-to-Sequence Models for Hebrew NLP

**Authors:** [Matan Eyal](https://md.mova.elamur.ai/free/en/docs/papers/authors/matan-eyal/); Hila Noga; [Roee Aharoni](https://md.mova.elamur.ai/free/en/docs/papers/authors/roee-aharoni/); [Idan Szpektor](https://md.mova.elamur.ai/free/en/docs/papers/authors/idan-szpektor/); [Reut Tsarfaty](https://md.mova.elamur.ai/free/en/docs/papers/authors/reut-tsarfaty/) · **Year:** 2023 · **Venue:** Findings of ACL 2023, pp. 7700–7708 **Link:** [aclanthology.org/2023.findings-acl.487](https://aclanthology.org/2023.findings-acl.487/) · library `findings-2023-eyal-multilingual-sequence-sequence-models-hebrew` · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) (ACL Anthology)

[**5** citations](https://openalex.org/works?filter=cites:W4385572234) openalex · 2026-10-18[**4** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/findings-2023-eyal-multilingual-sequence-sequence-models-hebrew/) of the 140673 papers whose bibliographies we read[Findings of the Association for Computational Linguistics: ACL 2023](https://aclanthology.org/volumes/2023.findings-acl/)[licence](https://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://aclanthology.org/2023.findings-acl.487/)
- [PDF](https://aclanthology.org/2023.findings-acl.487.pdf)
- [DOI 10.18653/v1/2023.findings-acl.487](https://doi.org/10.18653/v1/2023.findings-acl.487)
- [OpenAlex](https://openalex.org/W4385572234)

**Authors (5)**

1. [Matan Eyal](https://openalex.org/A5035283714) h 10 · 1507 citations · 30 works
2. [Hila Noga](https://openalex.org/A5079607831) h 4 · 1269 citations · 9 works
3. [Roee Aharoni](https://openalex.org/A5027223654) h 23 · 1918 citations · 69 works
4. [Idan Szpektor](https://openalex.org/A5026091724) [ORCID](https://orcid.org/0009-0002-2746-4027) · h 30 · 2655 citations · 145 works
5. [Reut Tsarfaty](https://openalex.org/A5063283689) [ORCID](https://orcid.org/0000-0001-6406-6568) · h 24 · 2886 citations · 217 works

Citations and author metrics from openalex (2026-10-18); authors with the same name are not always told apart.

## What it is about

The authors argue that Hebrew encoder-only models (AlephBERT, AlephBERT-Gimmel, HeBERT) cannot emit morphemes or labels that are not in the input, so they need extra segment-then-tag components. They recast Hebrew tasks as text-to-text problems for multilingual mT5, in sizes from 300M to 13B parameters: QA on ParaShoot, NER (NEMO), sentiment, segmentation, POS tagging and lemmatisation. Output marks morpheme boundaries with "@@" and separates a morpheme from its tag with "»". The largest model reaches 77.5 F1 on ParaShoot (AlephBERT about 50), 98.7 segmentation and 97.5 POS in morpheme-aligned multiset F1 (ABG 98.1 and 96.2), and about 95.5 lemmatisation against 78.6 for YAP. The sentiment gain is small, and a manual review finds that 34% of the model's errors are annotation errors. The authors flag that mT5 is trained on 101 languages, which may hurt Hebrew.

## Ideas

- I1. Score Mova's segmenter and tagger with the morpheme-aligned multiset measure (aligned bag of morpheme plus tag pairs), not token accuracy, because one token can be several morphemes. Use the "@@" and "»" notation as a debug view of the segmenter output.
- I2. Before trusting a gold set, sample its disagreements: the review of the sentiment set shows a third of errors are label errors. Add this check to the Hebrew treebank intake.
- I3. Use mT5 only as a teacher: its segmentations of unpointed Hebrew could seed silver labels for the lexicon tables of the segmenter, checked against the gold treebank. The model never runs at test time.

## For Mova

I1 goes into the evaluation of `tools/hbo` and into the output notes of [11. Modern Hebrew in Universal Dependencies](https://md.mova.elamur.ai/free/en/docs/languages/he/11-ud-analysis/). I2 belongs to the intake of `treebanks/he/`. I3 is a teacher idea for `tools/hbo`, to be measured before any use.

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.18653/v1/2023.findings-acl.487](https://doi.org/10.18653/v1/2023.findings-acl.487)

```
@inproceedings{eyal2023multilingual,
  title = {{Multilingual Sequence-to-Sequence Models for Hebrew NLP}},
  author = {Eyal, Matan and Noga, Hila and Aharoni, Roee and Szpektor, Idan and Tsarfaty, Reut},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2023},
  year = {2023},
  doi = {10.18653/v1/2023.findings-acl.487},
  url = {https://doi.org/10.18653/v1/2023.findings-acl.487}
}
```

- [← previous HeQ: a Large and Diverse Hebrew Reading Comprehension Benchmark](https://md.mova.elamur.ai/free/en/docs/papers/findings-2023-cohen-heq-large-diverse-hebrew-reading/)
- [next → The Truth, The Whole Truth, and Nothing but the Truth: A New Benchmark Dataset for Hebrew Text Credibility Assessment](https://md.mova.elamur.ai/free/en/docs/papers/findings-2023-hagag-truth-whole-truth-nothing-but/)
- [↑ back to Modern Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/he/)

## References

31 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Adam Amram, Anat Ben David, and Reut Tsarfaty. 2018. Representations and architectures in neural sentiment analysis for morphologically rich languages: A case study from Modern Hebrew. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2242–2252, Santa Fe, New Mexico, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Adam+Amram%2C+Anat+Ben+David%2C+and+Reut+Tsarfaty%2E+2018%2E+Representations+and+architectures+in+neural+sentiment+analysis+for+morphologically+rich+languages%3A+A+case+study+from+Modern+Hebrew%2E+In+Proceedings+) [our card](https://md.mova.elamur.ai/free/en/docs/papers/coling-2018-amram-representations-architectures-neural-sentiment-analysis/)
2. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. [find](https://scholar.google.com/scholar?q=Wissam+Antoun%2C+Fady+Baly%2C+and+Hazem+Hajj%2E+2020%2E+AraBERT%3A+Transformer%2Dbased+model+for+Arabic+language+understanding%2E+In+Proceedings+of+the+4th+Workshop+on+Open%2DSource+Arabic+Corpora+and+Processing+Tool)
3. Dogu Araci. 2019. Finbert: Financial sentiment analysis with pre-trained language models. arXiv preprint arXiv:1908.10063. [↗](https://arxiv.org/abs/1908.10063)
4. Dan Bareket and Reut Tsarfaty. 2021. Neural modeling for named entities and morphology (NEMO2). Transactions of the Association for Computational Linguistics, 9:909–928. [find](https://scholar.google.com/scholar?q=Dan+Bareket+and+Reut+Tsarfaty%2E+2021%2E+Neural+modeling+for+named+entities+and+morphology+%28NEMO2%29%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+9%3A909%E2%80%93928%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/tacl-2021-bareket-neural-modeling-named-entities-morphology/)
5. Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. SciB-ERT: A pretrained language model for scientific text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3615– 3620, Hong Kong, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Iz+Beltagy%2C+Kyle+Lo%2C+and+Arman+Cohan%2E+2019%2E+SciB%2DERT%3A+A+pretrained+language+model+for+scientific+text%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+the)
6. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901. [find](https://scholar.google.com/scholar?q=Tom+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+et+al%2E+2020%2E+Language+models+are+few%2Dshot+lear)
7. Idan Brusilovsky and Reut Tsarfaty. 2022. Neural token segmentation for high token-internal complexity. arXiv preprint arXiv:2203.10845. [↗](https://arxiv.org/abs/2203.10845)
8. Branden Chan, Stefan Schweter, and Timo Möller. 2020. German’s next language model. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6788–6796, Barcelona, Spain (Online). International Committee on Computational Linguistics. [find](https://scholar.google.com/scholar?q=Branden+Chan%2C+Stefan+Schweter%2C+and+Timo+M%C3%B6ller%2E+2020%2E+German%E2%80%99s+next+language+model%2E+In+Proceedings+of+the+28th+International+Conference+on+Computational+Linguistics%2C+pages+6788%E2%80%936796%2C+Barcelona%2C+Spain+)
9. Avihay Chriqui and Inbal Yahav. 2022. Hebert & hebemo: a hebrew bert model and a tool for polarity analysis and emotion recognition. INFORMS Journal on Data Science. [find](https://scholar.google.com/scholar?q=Avihay+Chriqui+and+Inbal+Yahav%2E+2022%2E+Hebert+%26+hebemo%3A+a+hebrew+bert+model+and+a+tool+for+polarity+analysis+and+emotion+recognition%2E+INFORMS+Journal+on+Data+Science%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/langstream-2021-chriqui-hebert-hebemo-hebrew-bert-model/)
10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
11. Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. 2021. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1286–1305, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jesse+Dodge%2C+Maarten+Sap%2C+Ana+Marasovic%CC%81%2C+William+Agnew%2C+Gabriel+Ilharco%2C+Dirk+Groeneveld%2C+Margaret+Mitchell%2C+and+Matt+Gardner%2E+2021%2E+Documenting+large+webtext+corpora%3A+A+case+study+on+the+colossal+cl)
12. Eylon Guetta, Avi Shmidman, Shaltiel Shmidman, Cheyn Shmuel Shmidman, Joshua Guedalia, Moshe Koppel, Dan Bareket, Amit Seker, and Reut Tsarfaty. 2022. Large pre-trained models with extra-large vocabularies: A contrastive analysis of hebrew bert models and a new one to outperform them all. [find](https://scholar.google.com/scholar?q=Eylon+Guetta%2C+Avi+Shmidman%2C+Shaltiel+Shmidman%2C+Cheyn+Shmuel+Shmidman%2C+Joshua+Guedalia%2C+Moshe+Koppel%2C+Dan+Bareket%2C+Amit+Seker%2C+and+Reut+Tsarfaty%2E+2022%2E+Large+pre%2Dtrained+models+with+extra%2Dlarge+vocabul) [our card](https://md.mova.elamur.ai/free/en/docs/papers/hbo-2022-gueta-large-pre-trained-models-extra/)
13. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. 2022. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556. [↗](https://arxiv.org/abs/2203.15556)
14. Or Honovich, Roee Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, and Yossi Matias. 2022. TRUE: Re-evaluating factual consistency evaluation. In Proceedings of the Second DialDoc Workshop on Document-grounded Dialogue and Conversational Question Answering, pages 161–175, Dublin, Ireland. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Or+Honovich%2C+Roee+Aharoni%2C+Jonathan+Herzig%2C+Hagai+Taitelbaum%2C+Doron+Kukliansy%2C+Vered+Cohen%2C+Thomas+Scialom%2C+Idan+Szpektor%2C+Avinatan+Hassidim%2C+and+Yossi+Matias%2E+2022%2E+TRUE%3A+Re%2Devaluating+factual+consis)
15. Or Honovich, Leshem Choshen, Roee Aharoni, Ella Neeman, Idan Szpektor, and Omri Abend. 2021. q 2 : Evaluating factual consistency in knowledge-grounded dialogues via question generation and question answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7856–7870, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Or+Honovich%2C+Leshem+Choshen%2C+Roee+Aharoni%2C+Ella+Neeman%2C+Idan+Szpektor%2C+and+Omri+Abend%2E+2021%2E+q+2+%3A+Evaluating+factual+consistency+in+knowledge%2Dgrounded+dialogues+via+question+generation+and+question+a)
16. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361. [↗](https://arxiv.org/abs/2001.08361)
17. Omri Keren, Tal Avinari, Reut Tsarfaty, and Omer Levy. 2022. Breaking character: Are subwords good enough for mrls after all? arXiv preprint arXiv:2204.04748. [↗](https://arxiv.org/abs/2204.04748)
18. Omri Keren and Omer Levy. 2021. ParaShoot: A Hebrew question answering dataset. In Proceedings of the 3rd Workshop on Machine Reading for Question Answering, pages 106–112, Punta Cana, Dominican Republic. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Omri+Keren+and+Omer+Levy%2E+2021%2E+ParaShoot%3A+A+Hebrew+question+answering+dataset%2E+In+Proceedings+of+the+3rd+Workshop+on+Machine+Reading+for+Question+Answering%2C+pages+106%E2%80%93112%2C+Punta+Cana%2C+Dominican+Repub) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2021-keren-parashoot-hebrew-question-answering-dataset/)
19. Naama Ben Mordecai and Michael Elhadad. 2005. Hebrew named entity recognition. MONEY, 81(83.93):82–49. [find](https://scholar.google.com/scholar?q=Naama+Ben+Mordecai+and+Michael+Elhadad%2E+2005%2E+Hebrew+named+entity+recognition%2E+MONEY%2C+81%2883%2E93%29%3A82%E2%80%9349%2E)
20. Amir More, Amit Seker, Victoria Basmova, and Reut Tsarfaty. 2019. Joint transition-based models for morpho-syntactic parsing: Parsing strategies for MRLs and a case study from Modern Hebrew. Transactions of the Association for Computational Linguistics, 7:33–48. [find](https://scholar.google.com/scholar?q=Amir+More%2C+Amit+Seker%2C+Victoria+Basmova%2C+and+Reut+Tsarfaty%2E+2019%2E+Joint+transition%2Dbased+models+for+morpho%2Dsyntactic+parsing%3A+Parsing+strategies+for+MRLs+and+a+case+study+from+Modern+Hebrew%2E+Transacti) [our card](https://md.mova.elamur.ai/free/en/docs/papers/tacl-2019-more-joint-transition-based-models-morpho/)
21. Pedro Javier Ortiz Suárez, Laurent Romary, and Benoît Sagot. 2020. A monolingual approach to contextualized word embeddings for mid-resource languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1703–1714, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Pedro+Javier+Ortiz+Su%C3%A1rez%2C+Laurent+Romary%2C+and+Beno%C3%AEt+Sagot%2E+2020%2E+A+monolingual+approach+to+contextualized+word+embeddings+for+mid%2Dresource+languages%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the)
22. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2463–2473, Hong Kong, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Fabio+Petroni%2C+Tim+Rockt%C3%A4schel%2C+Sebastian+Riedel%2C+Patrick+Lewis%2C+Anton+Bakhtin%2C+Yuxiang+Wu%2C+and+Alexander+Miller%2E+2019%2E+Language+models+as+knowledge+bases%3F+In+Proceedings+of+the+2019+Conference+on+Emp)
23. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, et al. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(140):1–67. [find](https://scholar.google.com/scholar?q=Colin+Raffel%2C+Noam+Shazeer%2C+Adam+Roberts%2C+Katherine+Lee%2C+Sharan+Narang%2C+Michael+Matena%2C+Yanqi+Zhou%2C+Wei+Li%2C+Peter+J+Liu%2C+et+al%2E+2020%2E+Exploring+the+limits+of+transfer+learning+with+a+unified+text%2Dto%2Dt)
24. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Pranav+Rajpurkar%2C+Jian+Zhang%2C+Konstantin+Lopyrev%2C+and+Percy+Liang%2E+2016%2E+SQuAD%3A+100%2C000%2B+questions+for+machine+comprehension+of+text%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natu)
25. Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. 2018. Object hallucination in image captioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4035–4045, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Anna+Rohrbach%2C+Lisa+Anne+Hendricks%2C+Kaylee+Burns%2C+Trevor+Darrell%2C+and+Kate+Saenko%2E+2018%2E+Object+hallucination+in+image+captioning%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+Natural)
26. Katherine Lee, Orhan Firat, Ashish Agarwal, Clara Fannjiang, and David Sussillo. 2018. Hallucinations in neural machine translation. NeurIPS 2018 Workshop on Interpretability and Robustness for Audio, Speech, and Language. [find](https://scholar.google.com/scholar?q=Katherine+Lee%2C+Orhan+Firat%2C+Ashish+Agarwal%2C+Clara+Fannjiang%2C+and+David+Sussillo%2E+2018%2E+Hallucinations+in+neural+machine+translation%2E+NeurIPS+2018+Workshop+on+Interpretability+and+Robustness+for+Audio%2C)
27. Shoval Sade, Amit Seker, and Reut Tsarfaty. 2018. The Hebrew Universal Dependency treebank: Past present and future. In Proceedings of the Second Workshop on Universal Dependencies (UDW 2018), pages 133–143, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Shoval+Sade%2C+Amit+Seker%2C+and+Reut+Tsarfaty%2E+2018%2E+The+Hebrew+Universal+Dependency+treebank%3A+Past+present+and+future%2E+In+Proceedings+of+the+Second+Workshop+on+Universal+Dependencies+%28UDW+2018%29%2C+pages+1) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2018-sade-hebrew-universal-dependency-treebank-past/)
28. Louis Martin, Benjamin Muller, Pedro Javier Ortiz Suárez, Yoann Dupont, Laurent Romary, Éric de la Clergerie, Djamé Seddah, and Benoît Sagot. 2020. CamemBERT: a tasty French language model. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7203–7219, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Louis+Martin%2C+Benjamin+Muller%2C+Pedro+Javier+Ortiz+Su%C3%A1rez%2C+Yoann+Dupont%2C+Laurent+Romary%2C+%C3%89ric+de+la+Clergerie%2C+Djam%C3%A9+Seddah%2C+and+Beno%C3%AEt+Sagot%2E+2020%2E+CamemBERT%3A+a+tasty+French+language+model%2E+In+Proceed)
29. Amit Seker, Elron Bandel, Dan Bareket, Idan Brusilovsky, Refael Greenfeld, and Reut Tsarfaty. 2022. AlephBERT: Language model pre-training and evaluation from sub-word to sentence level. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 46–56, Dublin, Ireland. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Amit+Seker%2C+Elron+Bandel%2C+Dan+Bareket%2C+Idan+Brusilovsky%2C+Refael+Greenfeld%2C+and+Reut+Tsarfaty%2E+2022%2E+AlephBERT%3A+Language+model+pre%2Dtraining+and+evaluation+from+sub%2Dword+to+sentence+level%2E+In+Proceeding) [our card](https://md.mova.elamur.ai/free/en/docs/papers/acl-2022-seker-alephbert-language-model-pre-training/)
30. Reut Tsarfaty, Dan Bareket, Stav Klein, and Amit Seker. 2020. From SPMRL to NMRL: What did we learn (and unlearn) in a decade of parsing morphologically-rich languages (MRLs)? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7396– 7408, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Reut+Tsarfaty%2C+Dan+Bareket%2C+Stav+Klein%2C+and+Amit+Seker%2E+2020%2E+From+SPMRL+to+NMRL%3A+What+did+we+learn+%28and+unlearn%29+in+a+decade+of+parsing+morphologically%2Drich+languages+%28MRLs%29%3F+In+Proceedings+of+the+58)
31. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mT5: A massively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 483–498, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Linting+Xue%2C+Noah+Constant%2C+Adam+Roberts%2C+Mihir+Kale%2C+Rami+Al%2DRfou%2C+Aditya+Siddhant%2C+Aditya+Barua%2C+and+Colin+Raffel%2E+2021%2E+mT5%3A+A+massively+multilingual+pre%2Dtrained+text%2Dto%2Dtext+transformer%2E+In+Procee)