[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/aclbulk-2019-kishky-constrained-sequence-sequence-semitic-root/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [Ancient Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/hbo/) / Constrained Sequence-to-sequence Semitic Root Extraction for Enriching Word Embeddings

# Constrained Sequence-to-sequence Semitic Root Extraction for Enriching Word Embeddings

**Authors:** Ahmed El-Kishky; Xingyu Fu; Aseel Addawood; Nahil Sobh; Clare Voss; Jiawei Han · **Year:** 2019 · **Venue:** Proceedings of the Fourth Arabic Natural Language Processing Workshop **Link:** [aclanthology.org/W19-4610](https://aclanthology.org/W19-4610/) · library `aclbulk-2019-kishky-constrained-sequence-sequence-semitic-root` · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)

[**2** citations](https://openalex.org/works?filter=cites:W2970252926) openalex · 2026-10-18[**4** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/aclbulk-2019-kishky-constrained-sequence-sequence-semitic-root/) of the 140673 papers whose bibliographies we read[Proceedings of the Fourth Arabic Natural Language Processing Workshop](https://aclanthology.org/volumes/W19-46/)[licence](https://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://aclanthology.org/W19-4610/)
- [PDF](https://aclanthology.org/W19-4610.pdf)
- [DOI 10.18653/v1/w19-4610](https://doi.org/10.18653/v1/w19-4610)
- [OpenAlex](https://openalex.org/W2970252926)

**Authors (6)**

1. Ahmed El-Kishky
2. [Xingyu Fu](https://openalex.org/A5102854017) [ORCID](https://orcid.org/0000-0002-2649-8535) · h 12 · 591 citations · 39 works
3. [Aseel Addawood](https://openalex.org/A5034426215) h 9 · 370 citations · 22 works
4. [Nahil Sobh](https://openalex.org/A5016018855) [ORCID](https://orcid.org/0000-0001-6952-3465) · h 25 · 3121 citations · 349 works
5. [Clare Voss](https://openalex.org/A5009361768) [ORCID](https://orcid.org/0000-0001-5023-6474) · h 23 · 2832 citations · 141 works
6. [Jiawei Han](https://openalex.org/A5019539533) [ORCID](https://orcid.org/0000-0002-3629-2696) · h 146 · 95944 citations · 1115 works

Citations and author metrics from openalex (2026-10-18); authors with the same name are not always told apart.

## What it is about

The authors extract the root of a Semitic word with a sequence-to-sequence model (a BiGRU encoder and a GRU decoder over characters). The novelty is constrained beam search: a trie built from the dictionary of valid roots prunes every decoding step, so the output is always a real root. The model is compared with multiclass classifiers, unconstrained seq2seq models and rule-based stemmers. Five-fold accuracy is 98.8% on a Hebrew word-root list of about 11,500 words (roughly 500 roots, Wiktionary-derived) and 83.2% on an Arabic list of 140K words (11K roots). On rare Arabic roots (three or fewer training occurrences) and on unseen roots the constrained model still does best. The root is then used to split words into prefix, stem and suffix slots, and these slots enrich FastText-style embeddings, which helps on morphemic analogies and word similarity.

## Ideas

- I1. Trie-constrained output as a plain lookup: a candidate root is accepted only if it is in the root table, and otherwise the analyser keeps all candidates and reports the word as unknown. For Hebrew, with a few hundred roots, this is a finite-state check that costs nothing and can be explained.
- I2. Stem and slot rule: the stem is the shortest contiguous span that contains the root in order; letters outside it are prefixes and suffixes, and letters that fall inside the gaps of the root are infixes. For Hebrew, write the analogous templates from the grammar (the binyanim and the weak-verb patterns), not by copying the Arabic slots.
- I3. A test set for root extraction: split Hebrew roots by frequency (rare, unseen) and report accuracy on each group, as the paper does, before adding any learned component to `tools/hbo`.

## For Mova

I1 is a rule for [The verb](https://md.mova.elamur.ai/free/en/docs/languages/hbo/06-verb/) and [The Noun, the Adjective and the Numeral](https://md.mova.elamur.ai/free/en/docs/languages/hbo/04-noun/) (root check); I2 belongs in [The weak verbs](https://md.mova.elamur.ai/free/en/docs/languages/hbo/07-weak-verbs/) and [The Noun, the Adjective and the Numeral](https://md.mova.elamur.ai/free/en/docs/languages/hbo/04-noun/); I3 is a measure for `tools/hbo`. The Wiktionary-derived root list is CC BY-SA data: it may be tested against and learned from, but it is not carried into a release.

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.18653/v1/w19-4610](https://doi.org/10.18653/v1/w19-4610)

```
@inproceedings{elkishky2019constrained,
  title = {{Constrained Sequence-to-sequence Semitic Root Extraction for Enriching Word Embeddings}},
  author = {El-Kishky, Ahmed and Fu, Xingyu and Addawood, Aseel and Sobh, Nahil and Voss, Clare and Han, Jiawei},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Workshop},
  year = {2019},
  doi = {10.18653/v1/w19-4610},
  url = {https://doi.org/10.18653/v1/w19-4610}
}
```

- [← previous A Characterwise Windowed Approach to Hebrew Morphological Segmentation](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2018-zeldes-characterwise-windowed-approach-hebrew-morphological/)
- [next → What's Wrong with Hebrew NLP? And How to Make it Right](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2019-tsarfaty-what-s-wrong-hebrew-nlp/)
- [↑ back to Ancient Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/hbo/)

## References

30 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yasir Alhanini and Mohd Juzaiddin Ab Aziz. 2011. The enhancement of arabic stemming by using light stemming and dictionary-based stemming. JSEA. [find](https://scholar.google.com/scholar?q=Yasir+Alhanini+and+Mohd+Juzaiddin+Ab+Aziz%2E+2011%2E+The+enhancement+of+arabic+stemming+by+using+light+stemming+and+dictionary%2Dbased+stemming%2E+JSEA%2E)
2. Mohammed Aljlayl and Ophir Frieder. 2002. On arabic search: improving the retrieval effectiveness via a light stemming approach. In CIKM. [find](https://scholar.google.com/scholar?q=Mohammed+Aljlayl+and+Ophir+Frieder%2E+2002%2E+On+arabic+search%3A+improving+the+retrieval+effectiveness+via+a+light+stemming+approach%2E+In+CIKM%2E)
3. Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. 2017. Guided open vocabulary image captioning with constrained beam search. In EMNLP. [find](https://scholar.google.com/scholar?q=Peter+Anderson%2C+Basura+Fernando%2C+Mark+Johnson%2C+and+Stephen+Gould%2E+2017%2E+Guided+open+vocabulary+image+captioning+with+constrained+beam+search%2E+In+EMNLP%2E)
4. Andras Kornai. 2007. Mathematical linguistics. Springer Science & Business Media. [find](https://scholar.google.com/scholar?q=Andras+Kornai%2E+2007%2E+Mathematical+linguistics%2E+Springer+Science+%26+Business+Media%2E)
5. Abderrahim Boudlal, Mohamed Ould Abdallahi Ould Bebah, Abdelhak Lakhouaja, Azzeddine Mazroui, and Abdelouafi Meziane. 2011. A markovian approach for arabic root extraction. Int. Arab J. Inf. Technol. [find](https://scholar.google.com/scholar?q=Abderrahim+Boudlal%2C+Mohamed+Ould+Abdallahi+Ould+Bebah%2C+Abdelhak+Lakhouaja%2C+Azzeddine+Mazroui%2C+and+Abdelouafi+Meziane%2E+2011%2E+A+markovian+approach+for+arabic+root+extraction%2E+Int%2E+Arab+J%2E+Inf%2E+Technol%2E)
6. Leah S Larkey, Lisa Ballesteros, and Margaret E Connell. 2002. Improving stemming for arabic information retrieval: light stemming and co-occurrence analysis. In ACM SIGIR. [find](https://scholar.google.com/scholar?q=Leah+S+Larkey%2C+Lisa+Ballesteros%2C+and+Margaret+E+Connell%2E+2002%2E+Improving+stemming+for+arabic+information+retrieval%3A+light+stemming+and+co%2Doccurrence+analysis%2E+In+ACM+SIGIR%2E)
7. Yaacov Choueka. 1990. Mlim-a system for full, exact, on-line grammatical analysis of modern hebrew. In ICCE. [find](https://scholar.google.com/scholar?q=Yaacov+Choueka%2E+1990%2E+Mlim%2Da+system+for+full%2C+exact%2C+on%2Dline+grammatical+analysis+of+modern+hebrew%2E+In+ICCE%2E)
8. Yoong Keok Lee, Aria Haghighi, and Regina Barzilay. 2011. Modeling syntactic context improves morphological segmentation. In CoNLL. [find](https://scholar.google.com/scholar?q=Yoong+Keok+Lee%2C+Aria+Haghighi%2C+and+Regina+Barzilay%2E+2011%2E+Modeling+syntactic+context+improves+morphological+segmentation%2E+In+CoNLL%2E)
9. Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555. [↗](https://arxiv.org/abs/1412.3555)
10. Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015. Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025. [↗](https://arxiv.org/abs/1508.04025)
11. Kareem Darwish. 2002. Building a shallow arabic morphological analyzer in one day. In ACL-02 workshop on Computational approaches to semitic languages. [find](https://scholar.google.com/scholar?q=Kareem+Darwish%2E+2002%2E+Building+a+shallow+arabic+morphological+analyzer+in+one+day%2E+In+ACL%2D02+workshop+on+Computational+approaches+to+semitic+languages%2E)
12. Mohanned Momani and Jamil Faraj. 2007. A novel algorithm to extract tri-literal arabic roots. In Computer Systems and Applications, 2007. AICCSA’07. [find](https://scholar.google.com/scholar?q=Mohanned+Momani+and+Jamil+Faraj%2E+2007%2E+A+novel+algorithm+to+extract+tri%2Dliteral+arabic+roots%2E+In+Computer+Systems+and+Applications%2C+2007%2E+AICCSA%E2%80%9907%2E)
13. Ezra Daya, Dan Roth, and Shuly Wintner. 2004. Learning hebrew roots: Machine learning with linguistic constraints. In EMNLP. [find](https://scholar.google.com/scholar?q=Ezra+Daya%2C+Dan+Roth%2C+and+Shuly+Wintner%2E+2004%2E+Learning+hebrew+roots%3A+Machine+learning+with+linguistic+constraints%2E+In+EMNLP%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2004-daya-learning-hebrew-roots-machine-learning/)
14. Armand Joulin Piotr Bojanowski, Edouard Grave and Tomas Mikolov. 2017. Enriching word vectors with subword information. TACL. [find](https://scholar.google.com/scholar?q=Armand+Joulin+Piotr+Bojanowski%2C+Edouard+Grave+and+Tomas+Mikolov%2E+2017%2E+Enriching+word+vectors+with+subword+information%2E+TACL%2E)
15. Ezra Daya, Dan Roth, and Shuly Wintner. 2008. Identifying semitic roots: Machine learning with linguistic constraints. Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ezra+Daya%2C+Dan+Roth%2C+and+Shuly+Wintner%2E+2008%2E+Identifying+semitic+roots%3A+Machine+learning+with+linguistic+constraints%2E+Computational+Linguistics%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/cl-2008-daya-identifying-semitic-roots-machine-learning/)
16. Paul Rodrigues and Damir Cavar. 2007. Learning arabic morphology using statistical constraint-satisfaction models. Amsterdam studies in the theory and history of linguistic science. Series 4. [find](https://scholar.google.com/scholar?q=Paul+Rodrigues+and+Damir+Cavar%2E+2007%2E+Learning+arabic+morphology+using+statistical+constraint%2Dsatisfaction+models%2E+Amsterdam+studies+in+the+theory+and+history+of+linguistic+science%2E+Series+4%2E)
17. Ahmed El-Kishky, Frank Xu, Aston Zhang, Stephen Macke, and Jiawei Han. 2018. Entropy-based subword mining with an application to word embeddings. In SCLeM 2018. [find](https://scholar.google.com/scholar?q=Ahmed+El%2DKishky%2C+Frank+Xu%2C+Aston+Zhang%2C+Stephen+Macke%2C+and+Jiawei+Han%2E+2018%2E+Entropy%2Dbased+subword+mining+with+an+application+to+word+embeddings%2E+In+SCLeM+2018%2E)
18. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In NIPS. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+NIPS%2E)
19. Khaled Elghamry. 2005. A constraint-based algorithm for the identification of arabic roots. In Proceedings of the Midwest Computational Linguistics Colloquium. Indiana University. Bloomington, IN. [find](https://scholar.google.com/scholar?q=Khaled+Elghamry%2E+2005%2E+A+constraint%2Dbased+algorithm+for+the+identification+of+arabic+roots%2E+In+Proceedings+of+the+Midwest+Computational+Linguistics+Colloquium%2E+Indiana+University%2E+Bloomington%2C+IN%2E)
20. Kazem Taghva, Rania Elkhoury, and Jeffrey Coombs. 2005. Arabic stemming without a root dictionary. In Information Technology: Coding and Computing. [find](https://scholar.google.com/scholar?q=Kazem+Taghva%2C+Rania+Elkhoury%2C+and+Jeffrey+Coombs%2E+2005%2E+Arabic+stemming+without+a+root+dictionary%2E+In+Information+Technology%3A+Coding+and+Computing%2E)
21. Qussai Yaseen and Ismail Hmeidi. 2014. Extracting the roots of arabic words without removing affixes. Journal of Information Science. [find](https://scholar.google.com/scholar?q=Qussai+Yaseen+and+Ismail+Hmeidi%2E+2014%2E+Extracting+the+roots+of+arabic+words+without+removing+affixes%2E+Journal+of+Information+Science%2E)
22. Mohammed Elrazzaz, Shady Elbassuoni, Khaled Shaban, and Chadi Helwe. 2017. Methodical evaluation of arabic word embeddings. In ACL. [find](https://scholar.google.com/scholar?q=Mohammed+Elrazzaz%2C+Shady+Elbassuoni%2C+Khaled+Shaban%2C+and+Chadi+Helwe%2E+2017%2E+Methodical+evaluation+of+arabic+word+embeddings%2E+In+ACL%2E)
23. Muhammad Murtada al Zabidi and Sayyid Muhammad Murthada. 1886. Taj al-‘arus min jawahir al-qamus. Dar al-Hidayah. [find](https://scholar.google.com/scholar?q=Muhammad+Murtada+al+Zabidi+and+Sayyid+Muhammad+Murthada%2E+1886%2E+Taj+al%2D%E2%80%98arus+min+jawahir+al%2Dqamus%2E+Dar+al%2DHidayah%2E)
24. André Freitas, Siamak Barzegar, Juliano Efson Sales, Siegfried Handschuh, and Brian Davis. 2016. Semantic Relatedness for All (Languages): A Comparative Analysis of Multilingual Semantic Relatedness Using Machine Translation. [find](https://scholar.google.com/scholar?q=Andre%CC%81+Freitas%2C+Siamak+Barzegar%2C+Juliano+Efson+Sales%2C+Siegfried+Handschuh%2C+and+Brian+Davis%2E+2016%2E+Semantic+Relatedness+for+All+%28Languages%29%3A+A+Comparative+Analysis+of+Multilingual+Semantic+Relatedness+)
25. Taha Zerrouki. 2010. Tashaphyne, arabic light stemmer/segment. [find](https://scholar.google.com/scholar?q=Taha+Zerrouki%2E+2010%2E+Tashaphyne%2C+arabic+light+stemmer%2Fsegment%2E)
26. Yaoyuan Zhang, Zhenxu Ye, Yansong Feng, Dongyan Zhao, and Rui Yan. 2017. A constrained sequence-to-sequence neural model for sentence simplification. arXiv preprint arXiv:1704.02312. [↗](https://arxiv.org/abs/1704.02312)
27. Daniela Gerz, Ivan Vulić, Edoardo Ponti, Jason Naradowsky, Roi Reichart, and Anna Korhonen. 2018. Language modeling for morphologically rich languages: Character-aware modeling for word-level prediction. TACL. [find](https://scholar.google.com/scholar?q=Daniela+Gerz%2C+Ivan+Vulic%CC%81%2C+Edoardo+Ponti%2C+Jason+Naradowsky%2C+Roi+Reichart%2C+and+Anna+Korhonen%2E+2018%2E+Language+modeling+for+morphologically+rich+languages%3A+Character%2Daware+modeling+for+word%2Dlevel+predict)
28. Nizar Y Habash. 2010. Introduction to arabic natural language processing. Synthesis Lectures on Human Language Technologies. [find](https://scholar.google.com/scholar?q=Nizar+Y+Habash%2E+2010%2E+Introduction+to+arabic+natural+language+processing%2E+Synthesis+Lectures+on+Human+Language+Technologies%2E)
29. Shereen Khoja and Roger Garside. 1999. Stemming arabic text. Lancaster, UK, Computing Department, Lancaster University. [find](https://scholar.google.com/scholar?q=Shereen+Khoja+and+Roger+Garside%2E+1999%2E+Stemming+arabic+text%2E+Lancaster%2C+UK%2C+Computing+Department%2C+Lancaster+University%2E)
30. Yoon Kim. 2014. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882. [↗](https://arxiv.org/abs/1408.5882)