[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/aclbulk-2024-fono-embible-reconstruction-ancient-hebrew-aramaic/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [Ancient Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/hbo/) / Embible: Reconstruction of Ancient Hebrew and Aramaic Texts Using Transformers

# Embible: Reconstruction of Ancient Hebrew and Aramaic Texts Using Transformers

**Authors:** Niv Fono; Harel Moshayof; Eldar Karol; Itai Assraf; [Mark Last](https://md.mova.elamur.ai/free/en/docs/papers/authors/mark-last/) · **Year:** 2024 · **Venue:** Findings of the Association for Computational Linguistics: EACL 2024 **Link:** [aclanthology.org/2024.findings-eacl.56](https://aclanthology.org/2024.findings-eacl.56/) · library `aclbulk-2024-fono-embible-reconstruction-ancient-hebrew-aramaic` · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)

[**0** citations](https://openalex.org/works?filter=cites:W7126374875) openalex-snapshot · 2026-10-14[**3** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/aclbulk-2024-fono-embible-reconstruction-ancient-hebrew-aramaic/) of the 140673 papers whose bibliographies we read[Findings of the Association for Computational Linguistics: EACL 2024](https://aclanthology.org/volumes/2024.findings-eacl/)[licence](https://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://aclanthology.org/2024.findings-eacl.56/)
- [PDF](https://aclanthology.org/2024.findings-eacl.56.pdf)
- [DOI 10.18653/v1/2024.findings-eacl.56](https://doi.org/10.18653/v1/2024.findings-eacl.56)
- [OpenAlex](https://openalex.org/W7126374875)

**Authors (5)**

1. [Niv Fono](https://openalex.org/A5124606921) [Semantic Scholar](https://www.semanticscholar.org/author/2291362706) · h 0 · 0 citations · 5 works
2. [Harel Moshayof](https://openalex.org/A5124630893) [Semantic Scholar](https://www.semanticscholar.org/author/2291363940) · h 0 · 0 citations · 2 works
3. [Eldar Karol](https://openalex.org/A5124494450) [Semantic Scholar](https://www.semanticscholar.org/author/2291360891) · h 0 · 0 citations · 2 works
4. [Itai Assraf](https://openalex.org/A5120861915) [Semantic Scholar](https://www.semanticscholar.org/author/2291362704) · h 0 · 0 citations · 3 works
5. [Mark Last](https://openalex.org/A5066910344) [ORCID](https://orcid.org/0000-0003-0748-7918) · [Semantic Scholar](https://www.semanticscholar.org/author/3045152) · h 34 · 4781 citations · 275 works

Citations and author metrics from openalex-snapshot (2026-10-14); authors with the same name are not always told apart.

## What it is about

Damaged inscriptions are restored by masked-language-model prediction. The authors fine-tune TavBERT (character-level), mBERT, DistilBERT and AlephBERT-Gimmel on 22,144 Biblical verses, and test on 536 verses with 5%, 10% and 15% of characters or words masked. Their Ensemble combines TavBERT's character and whitespace predictions with the top word candidates of AlephBERT-Gimmel, and keeps only candidates that match both. With unknown whitespace, the Ensemble's WordHit@5 at 5% masking is 0.440, against 0.147 for AlephBERT-Gimmel alone (Table 2). Known whitespace raises all scores. The results are below the Akkadian reference, which the authors attribute to genre. The test uses masked Tanakh verses, not real inscriptions, and no Aramaic inscriptions were used in pretraining.

## Ideas

- I1. Constrained word completion: when the damaged word's length and boundaries are known from the font size and gaps, drop every candidate of the wrong length. This is a deterministic filter that can sit on top of any candidate list in `tools/ocrfix`.
- I2. Keep only candidates that two independent sources agree on (a character-level guess and a word-level guess). The intersection is a rule Mova can implement with BTreeMap ordering, and it can be run with a dictionary such as `macula dict` in place of a neural word model.
- I3. Treat word separation as its own checked decision, since known whitespace raised accuracy a lot. This matters for the unpointed mode of `tools/hbo`.
- I4. Adopt their masking protocol (5/10/15%, two masking strategies, fixed seed) as a reproducible test harness for `tools/ocrfix` and for the robustness of `tools/hbo`.

## For Mova

I1 and I2 apply to `tools/ocrfix` (correction of damaged printed editions). I3 applies to the unpointed mode of `tools/hbo` and to [1. Script and pointing](https://md.mova.elamur.ai/free/en/docs/languages/hbo/01-script/). I4 is a test set for both. The inscriptions are for [Biblical Aramaic texts, annotated](https://md.mova.elamur.ai/free/en/docs/languages/arc/texts/), where the paper's limitation (masked Bible, not inscriptions) is noted.

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.18653/v1/2024.findings-eacl.56](https://doi.org/10.18653/v1/2024.findings-eacl.56)

```
@inproceedings{fono2024embible,
  title = {{Embible: Reconstruction of Ancient Hebrew and Aramaic Texts Using Transformers}},
  author = {Fono, Niv and Moshayof, Harel and Karol, Eldar and Assraf, Itai and Last, Mark},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2024},
  year = {2024},
  doi = {10.18653/v1/2024.findings-eacl.56},
  url = {https://doi.org/10.18653/v1/2024.findings-eacl.56}
}
```

- [← previous Style Classification of Rabbinic Literature for Detection of Lost Midrash Tanhuma Material](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2022-tannor-style-classification-rabbinic-literature-detection/)
- [next → Detecting Narrative Patterns in Biblical Hebrew and Greek](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2024-mcgovern-detecting-narrative-patterns-biblical-hebrew/)
- [↑ back to Ancient Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/hbo/)

## References

17 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yannis Assael, Thea Sommerschield, and Jonathan Prag. 2019. Restoring ancient text using deep learning: a case study on Greek epigraphy. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6368–6375, Hong Kong, China. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/d19-1668)
2. Avihay Chriqui and Inbal Yahav. 2022. Hebert and hebemo: A hebrew bert model and a tool for polarity analysis and emotion recognition. INFORMS Journal on Data Science, 1(1):81–95. [↗](https://doi.org/10.1287/ijds.2022.0016) [our card](https://md.mova.elamur.ai/free/en/docs/papers/langstream-2021-chriqui-hebert-hebemo-hebrew-bert-model/)
3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
4. Akkadian texts: A masked language modelling approach. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4682–4691, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Akkadian+texts%3A+A+masked+language+modelling+approach%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+4682%E2%80%934691%2C+Online+and+Punta+Cana%2C+Dominican+Repub)
5. Jiaming Luo, Frederik Hartmann, Enrico Santus, Regina Barzilay, and Yuan Cao. 2021. Deciphering Undersegmented Ancient Scripts Using Phonetic Prior. Transactions of the Association for Computational Linguistics, 9:69–81. [↗](https://doi.org/10.1162/tacl_a_00354) [our card](https://md.mova.elamur.ai/free/en/docs/papers/tacl-2021-luo-deciphering-undersegmented-ancient-scripts-phonetic/)
6. Joana Ribeiro, Shashi Narayan, Shay Cohen, and Xavier Carreras. 2018. Local string transduction as sequence labeling. In 27th International Conference on Computational Linguistics, pages 1360–1371. Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=Joana+Ribeiro%2C+Shashi+Narayan%2C+Shay+Cohen%2C+and+Xavier+Carreras%2E+2018%2E+Local+string+transduction+as+sequence+labeling%2E+In+27th+International+Conference+on+Computational+Linguistics%2C+pages+1360%E2%80%931371%2E+As)
7. Shruti Rijhwani, Antonios Anastasopoulos, and Graham Neubig. 2020. Ocr post correction for endangered language texts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5931–5942. [↗](https://doi.org/10.18653/v1/2020.emnlp-main.478)
8. Sascha Rothe, Jonathan Mallinson, Eric Malmi, Sebastian Krause, and Aliaksei Severyn. 2021. A simple recipe for multilingual grammatical error correction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 702–707. [↗](https://doi.org/10.18653/v1/2021.acl-short.89)
9. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. [↗](https://arxiv.org/abs/1910.01108)
10. Samrat Dutta, Shreyansh Jain, Ayush Maheshwari, Souvik Pal, Ganesh Ramakrishnan, and Preethi Jyothi. 2022. Error correction in asr using sequence-to-sequence models. arXiv preprint arXiv:2202.01157. [↗](https://arxiv.org/abs/2202.01157)
11. William M Schniedewind. 2006. Aramaic, the death of written hebrew, and language shift in the persian period. Margins of Writing, Origins of Cultures, pages 137–147. [find](https://scholar.google.com/scholar?q=William+M+Schniedewind%2E+2006%2E+Aramaic%2C+the+death+of+written+hebrew%2C+and+language+shift+in+the+persian+period%2E+Margins+of+Writing%2C+Origins+of+Cultures%2C+pages+137%E2%80%93147%2E)
12. Ethan Fetaya, Yonatan Lifshitz, Elad Aaron, and Shai Gordin. 2020. Restoration of fragmentary babylonian texts using recurrent neural networks. Proceedings of the National Academy of Sciences, 117(37):22743–22751. [↗](https://doi.org/10.1073/pnas.2003794117) [our card](https://md.mova.elamur.ai/free/en/docs/papers/langstream-2020-fetaya-restoration-fragmentary-babylonian-texts-using/)
13. Judy Siegel-Itzkovich. 2023. Ancient tablet found on mount ebal predates known hebrew inscriptions. The Jerusalem Post. Avaialble at: [https://www.jpost.com/archaeology/](https://www.jpost.com/archaeology/) article-743039. 14 May 2023. [↗](https://www.jpost.com/archaeology/)
14. Eylon Guetta, Avi Shmidman, Shaltiel Shmidman, Cheyn Shmuel Shmidman, Joshua Guedalia, Moshe Koppel, Dan Bareket, Amit Seker, and Reut Tsarfaty. 2022. Large pre-trained models with extra-large vocabularies: A contrastive analysis of hebrew bert models and a new one to outperform them all. arXiv preprint arXiv:2211.15199. [↗](https://arxiv.org/abs/2211.15199) [our card](https://md.mova.elamur.ai/free/en/docs/papers/hbo-2022-gueta-large-pre-trained-models-extra/)
15. Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. 2022. ByT5: Towards a token-free future with pre-trained byte-to-byte models. Transactions of the Association for Computational Linguistics, 10:291–306. [↗](https://doi.org/10.1162/tacl_a_00461)
16. Omri Keren, Tal Avinari, Reut Tsarfaty, and Omer Levy. 2022. Breaking character: Are subwords good enough for mrls after all? arXiv preprint arXiv:2204.04748. [↗](https://arxiv.org/abs/2204.04748)
17. Koren Lazar, Benny Saret, Asaf Yehudai, Wayne Horowitz, Nathan Wasserman, and Gabriel Stanovsky. 2021. Filling the gaps in Ancient [find](https://scholar.google.com/scholar?q=Koren+Lazar%2C+Benny+Saret%2C+Asaf+Yehudai%2C+Wayne+Horowitz%2C+Nathan+Wasserman%2C+and+Gabriel+Stanovsky%2E+2021%2E+Filling+the+gaps+in+Ancient)