[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/aclbulk-2025-hatch-semitic-root-encoding-tokenization-based/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [Ancient Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/hbo/) / Semitic Root Encoding: Tokenization Based on the Templatic Morphology of Semitic Languages in NMT

# Semitic Root Encoding: Tokenization Based on the Templatic Morphology of Semitic Languages in NMT

**Authors:** Brendan T. Hatch; Stephen D. Richardson · **Year:** 2025 · **Venue:** Proceedings of The Third Arabic Natural Language Processing Conference, pp. 26–41 **Link:** [aclanthology.org/2025.arabicnlp-main.3](https://aclanthology.org/2025.arabicnlp-main.3/) · library `aclbulk-2025-hatch-semitic-root-encoding-tokenization-based` · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) (ACL Anthology)

[**0** citations](https://openalex.org/works?filter=cites:W4416037689) openalex · 2026-10-18[Proceedings of The Third Arabic Natural Language Processing Conference](https://aclanthology.org/volumes/2025.arabicnlp-main/)[licence](https://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://aclanthology.org/2025.arabicnlp-main.3/)
- [PDF](https://aclanthology.org/2025.arabicnlp-main.3.pdf)
- [DOI 10.18653/v1/2025.arabicnlp-main.3](https://doi.org/10.18653/v1/2025.arabicnlp-main.3)
- [OpenAlex](https://openalex.org/W4416037689)

**Authors (2)**

1. [Brendan T. Hatch](https://openalex.org/A5120309365) h 0 · 0 citations · 1 works
2. Stephen D. Richardson

Citations and author metrics from openalex (2026-10-18); authors with the same name are not always told apart.

## What it is about

BPE and unigram tokenizers split words only into continuous strings, so they hide the root-and-template structure of Semitic words. Semitic Root Encoding (SRE) first runs a morphological analyser (CAMeL Tools, calima-msar) on each word and rewrites the stem as a root token (for example <KTB>) plus a template-stem token (for example 12u3), then applies SentencePiece BPE. In English-to-Arabic NMT with a BART-style model, SRE gains an average of 1.15 BLEU over BPE across four trials (27.54 against 26.39), and a small human ranking of 100 sentences by three evaluators favours SRE. The gain disappears at a 20,000-token vocabulary (27.62 against 28.03). Ablations show that random "roots" (Fake-SRE) lose more than 3 BLEU against real roots, and whole-stem tokens (Stem-SRE) lose too, which suggests the model uses the root and template separately. Generated words with a root and template that do not combine in Arabic are rare (2 in 9,669 sentences in one trial, 0 in another).

## Ideas

- I1. A root-and-template table for `tools/hbo`: for each Hebrew lemma, store the root (consonant radicals) and the pattern (binyan and template) as two symbols, the way the paper's <KTB> + 12u3 pairs work. The `lex` and `pre` tables in the segmenter can hold these as integer IDs, not strings.
- I2. A dubious-stem check: generate root × pattern combinations and count how many occur in the MACULA lexicon. The paper's method (count the combinations that are not words, then review by hand) is a measure of how far the SLM over-generates in morphology.
- I3. A negative control for any root-based rule: replace the real roots with random consonant sets and check that the score falls. If it does not fall, the rule is memorising surface forms, not using morphology. This is the paper's Fake-SRE idea, and it fits the "a gate that cannot show red" rule.

## For Mova

I1 goes into [The Noun, the Adjective and the Numeral](https://md.mova.elamur.ai/free/en/docs/languages/hbo/04-noun/) and [The verb](https://md.mova.elamur.ai/free/en/docs/languages/hbo/06-verb/), and into the `lex` tables of `tools/hbo`. I2 is a measure for `tools/hbo` and `macula dict`. I3 is a control to add to any root-based test in `tools/hbo`. The NMT result itself does not apply, since Mova does not translate.

Topics: [Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/topics/machine-translation/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.18653/v1/2025.arabicnlp-main.3](https://doi.org/10.18653/v1/2025.arabicnlp-main.3)

```
@inproceedings{hatch2025semitic,
  title = {{Semitic Root Encoding: Tokenization Based on the Templatic Morphology of Semitic Languages in NMT}},
  author = {Hatch, Brendan T. and Richardson, Stephen D.},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-main.3},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-main.3}
}
```

- [← previous The Development of Hebrew in Antiquity – A Computational Linguistic Study](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2025-baitner-development-hebrew-antiquity-computational-linguistic/)
- [next → Integrating Semantic and Statistical Features for Authorial Clustering of Qumran Scrolls](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2025-lourie-integrating-semantic-statistical-features-authorial/)
- [↑ back to Ancient Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/hbo/)

## References

22 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. Farasa: A fast and furious segmenter for arabic. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: Demonstrations, pages 11–16. [find](https://scholar.google.com/scholar?q=Ahmed+Abdelali%2C+Kareem+Darwish%2C+Nadir+Durrani%2C+and+Hamdy+Mubarak%2E+2016%2E+Farasa%3A+A+fast+and+furious+segmenter+for+arabic%2E+In+Proceedings+of+the+2016+conference+of+the+North+American+chapter+of+the+asso) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2016-abdelali-farasa-fast-furious-segmenter-arabic/)
2. Mohamed Alkaoud and Mairaj Syed. 2020. On the importance of tokenization in arabic embedding models. In Proceedings of the fifth Arabic natural language processing workshop, pages 119–129. [find](https://scholar.google.com/scholar?q=Mohamed+Alkaoud+and+Mairaj+Syed%2E+2020%2E+On+the+importance+of+tokenization+in+arabic+embedding+models%2E+In+Proceedings+of+the+fifth+Arabic+natural+language+processing+workshop%2C+pages+119%E2%80%93129%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2020-alkaoud-importance-tokenization-arabic-embedding-models/)
3. Abdulrahman Almuhareb, Waleed Alsanie, and Abdulmohsen Al-Thubaity. 2019. Arabic word segmentation with long short-term memory neural networks and word embedding. IEEE Access, 7:12879–12887. [https://themqm.org/](https://themqm.org/) [↗](https://themqm.org/)
4. Manel Aloui, Hasna Chouikhi, Ghaith Chaabane, Haithem Kchaou, and Chehir Dhaouadi. 2024. 101 billion arabic words dataset. Preprint, arXiv:2405.01590. International Conference on Computational Linguistics: Technical Papers, pages 1398–1408, Osaka, Japan. [↗](https://arxiv.org/abs/2405.01590)
5. The COLING 2016 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2016+Organizing+Committee%2E)
6. Zaid Alyafeai, Maged S Al-shaibani, Mustafa Ghaleb, and Irfan Ahmad. 2023. Evaluating various tokenizers for arabic text classification. Neural Processing Letters, 55(3):2911–2933. [find](https://scholar.google.com/scholar?q=Zaid+Alyafeai%2C+Maged+S+Al%2Dshaibani%2C+Mustafa+Ghaleb%2C+and+Irfan+Ahmad%2E+2023%2E+Evaluating+various+tokenizers+for+arabic+text+classification%2E+Neural+Processing+Letters%2C+55%283%29%3A2911%E2%80%932933%2E)
7. Francisco Guzmán, Peng-Jen Chen, Myle Ott, Juan Pino, Guillaume Lample, Philipp Koehn, Vishrav Chaudhary, and Marc’Aurelio Ranzato. 2019. Two new evaluation datasets for low-resource machine translation: Nepali-english and sinhala-english. [find](https://scholar.google.com/scholar?q=Francisco+Guzm%C3%A1n%2C+Peng%2DJen+Chen%2C+Myle+Ott%2C+Juan+Pino%2C+Guillaume+Lample%2C+Philipp+Koehn%2C+Vishrav+Chaudhary%2C+and+Marc%E2%80%99Aurelio+Ranzato%2E+2019%2E+Two+new+evaluation+datasets+for+low%2Dresource+machine+translati)
8. Chantal Amrhein and Rico Sennrich. 2021. How suitable are subword segmentation strategies for translating non-concatenative morphology? In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 689–705, Punta Cana, Dominican Republic. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Chantal+Amrhein+and+Rico+Sennrich%2E+2021%2E+How+suitable+are+subword+segmentation+strategies+for+translating+non%2Dconcatenative+morphology%3F+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EM)
9. Taku Kudo. 2018. Subword regularization: Improving neural network translation models with multiple subword candidates. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 66–75, Melbourne, Australia. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Taku+Kudo%2E+2018%2E+Subword+regularization%3A+Improving+neural+network+translation+models+with+multiple+subword+candidates%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Li)
10. Aditi Chaudhary, Chunting Zhou, Lori Levin, Graham Neubig, David R. Mortensen, and Jaime Carbonell. 2018. Adapting word embeddings to new languages with morphological and phonological subword representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3285–3295, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Aditi+Chaudhary%2C+Chunting+Zhou%2C+Lori+Levin%2C+Graham+Neubig%2C+David+R%2E+Mortensen%2C+and+Jaime+Carbonell%2E+2018%2E+Adapting+word+embeddings+to+new+languages+with+morphological+and+phonological+subword+represen)
11. Taku Kudo and John Richardson. 2018. SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 66–71, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Taku+Kudo+and+John+Richardson%2E+2018%2E+SentencePiece%3A+A+simple+and+language+independent+subword+tokenizer+and+detokenizer+for+neural+text+processing%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+M)
12. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871–7880, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mike+Lewis%2C+Yinhan+Liu%2C+Naman+Goyal%2C+Marjan+Ghazvininejad%2C+Abdelrahman+Mohamed%2C+Omer+Levy%2C+Veselin+Stoyanov%2C+and+Luke+Zettlemoyer%2E+2020%2E+BART%3A+Denoising+sequence%2Dto%2Dsequence+pre%2Dtraining+for+natural+l)
13. Anne N. De Roeck and Waleed Al-Fares. 2000. A morphologically sensitive clustering algorithm for identifying Arabic roots. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics, pages 199–206, Hong Kong. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Anne+N%2E+De+Roeck+and+Waleed+Al%2DFares%2E+2000%2E+A+morphologically+sensitive+clustering+algorithm+for+identifying+Arabic+roots%2E+In+Proceedings+of+the+38th+Annual+Meeting+of+the+Association+for+Computationa) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2000-roeck-morphologically-sensitive-clustering-algorithm-identifying/)
14. Ahmed El-Kishky, Xingyu Fu, Aseel Addawood, Nahil Sobh, Clare Voss, and Jiawei Han. 2019. Constrained sequence-to-sequence Semitic root extraction for enriching word embeddings. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 88–96, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ahmed+El%2DKishky%2C+Xingyu+Fu%2C+Aseel+Addawood%2C+Nahil+Sobh%2C+Clare+Voss%2C+and+Jiawei+Han%2E+2019%2E+Constrained+sequence%2Dto%2Dsequence+Semitic+root+extraction+for+enriching+word+embeddings%2E+In+Proceedings+of+the+) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2019-kishky-constrained-sequence-sequence-semitic-root/)
15. Ossama Obeid, Nasser Zalmout, Salam Khalifa, Dima Taji, Mai Oudah, Bashar Alhafni, Go Inoue, Fadhl Eryani, Alexander Erdmann, and Nizar Habash. 2020. CAMeL tools: An open source python toolkit for Arabic natural language processing. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 7022–7032, Marseille, France. European Language Resources Association. [find](https://scholar.google.com/scholar?q=Ossama+Obeid%2C+Nasser+Zalmout%2C+Salam+Khalifa%2C+Dima+Taji%2C+Mai+Oudah%2C+Bashar+Alhafni%2C+Go+Inoue%2C+Fadhl+Eryani%2C+Alexander+Erdmann%2C+and+Nizar+Habash%2E+2020%2E+CAMeL+tools%3A+An+open+source+python+toolkit+for+Ara)
16. Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, Naman Goyal, Tom Birch, Vitaliy Liptchinsky, Sergey Edunov, Michael Auli, and Armand Joulin. 2021. Beyond english-centric multilingual machine translation. Journal of Machine Learning Research, 22(107):1–48. [find](https://scholar.google.com/scholar?q=Angela+Fan%2C+Shruti+Bhosale%2C+Holger+Schwenk%2C+Zhiyi+Ma%2C+Ahmed+El%2DKishky%2C+Siddharth+Goyal%2C+Mandeep+Baines%2C+Onur+Celebi%2C+Guillaume+Wenzek%2C+Vishrav+Chaudhary%2C+Naman+Goyal%2C+Tom+Birch%2C+Vitaliy+Liptchinsky%2C+S)
17. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+Annual+Meeting+of+the+Association+for+Comp) [our card](https://md.mova.elamur.ai/free/en/docs/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
18. Philip Gage. 1994. A new algorithm for data compression. The C Users Journal, 12(2):23–38. [find](https://scholar.google.com/scholar?q=Philip+Gage%2E+1994%2E+A+new+algorithm+for+data+compression%2E+The+C+Users+Journal%2C+12%282%29%3A23%E2%80%9338%2E)
19. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. MADAMIRA: A fast, comprehensive tool for morphological analysis and disambiguation of Arabic. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 1094–1101, Reykjavik, Iceland. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Arfath+Pasha%2C+Mohamed+Al%2DBadrashiny%2C+Mona+Diab%2C+Ahmed+El+Kholy%2C+Ramy+Eskander%2C+Nizar+Habash%2C+Manoj+Pooleery%2C+Owen+Rambow%2C+and+Ryan+Roth%2E+2014%2E+MADAMIRA%3A+A+fast%2C+comprehensive+tool+for+morphological+an) [our card](https://md.mova.elamur.ai/free/en/docs/papers/lingbulk-2014-pasha-madamira-fast-comprehensive-tool-morphological/)
20. Naman Goyal, Cynthia Gao, Vishrav Chaudhary, Peng-Jen Chen, Guillaume Wenzek, Da Ju, Sanjana Krishnan, Marc’Aurelio Ranzato, Francisco Guzmán, and Angela Fan. 2022. The Flores-101 evaluation benchmark for low-resource and multilingual machine translation. Transactions of the Association for Computational Linguistics, 10:522–538. [find](https://scholar.google.com/scholar?q=Naman+Goyal%2C+Cynthia+Gao%2C+Vishrav+Chaudhary%2C+Peng%2DJen+Chen%2C+Guillaume+Wenzek%2C+Da+Ju%2C+Sanjana+Krishnan%2C+Marc%E2%80%99Aurelio+Ranzato%2C+Francisco+Guzm%C3%A1n%2C+and+Angela+Fan%2E+2022%2E+The+Flores%2D101+evaluation+benchmark)
21. Francisco Guzmán, Houda Bouamor, Ramy Baly, and Nizar Habash. 2016. Machine translation evaluation for Arabic using morphologically-enriched embeddings. In Proceedings of COLING 2016, the 26th Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics. Evaluation (LREC’12), pages 2214–2218, Istanbul, Turkey. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Francisco+Guzm%C3%A1n%2C+Houda+Bouamor%2C+Ramy+Baly%2C+and+Nizar+Habash%2E+2016%2E+Machine+translation+evaluation+for+Arabic+using+morphologically%2Denriched+embeddings%2E+In+Proceedings+of+COLING+2016%2C+the+26th+Maja+Po) [our card](https://md.mova.elamur.ai/free/en/docs/papers/popovic-2015-chrf-character-n-gram-f/)
22. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Matt+Post%2E+2018%2E+A+call+for+clarity+in+reporting+BLEU+scores%2E+In+Proceedings+of+the+Third+Conference+on+Machine+Translation%3A+Research+Papers%2C+pages+186%E2%80%93+191%2C+Brussels%2C+Belgium%2E+Association+for+Computa) [our card](https://md.mova.elamur.ai/free/en/docs/papers/post-2018-call-clarity-reporting-bleu-scores/)