[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/akerman-2023-ebible-corpus-data-model-benchmarks/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/mtrans/) / The eBible Corpus: Data and Model Benchmarks for Bible Translation for Low-Resource Languages

# The eBible Corpus: Data and Model Benchmarks for Bible Translation for Low-Resource Languages

**Authors:** Vesa Akerman, David Baines, Damien Daspit, [Ulf Hermjakob](https://md.mova.elamur.ai/free/en/docs/papers/authors/ulf-hermjakob/), Taeho Jang, Colin Leong, Michael Martin, Joel Mathew, [Jonathan Robie](https://md.mova.elamur.ai/free/en/docs/papers/authors/jonathan-robie/), Marcus Schwarting · **Year:** 2023 · **Venue:** arXiv:2304.09919 **Link:** [arxiv.org/abs/2304.09919](https://arxiv.org/abs/2304.09919) · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)

[**0** citations](https://openalex.org/works?filter=cites:W4366732166) openalex · 2026-10-18[**9** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/akerman-2023-ebible-corpus-data-model-benchmarks/) of the 140673 papers whose bibliographies we read[arXiv:2304.09919](https://openalex.org/S4306400194)[licence](http://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://arxiv.org/abs/2304.09919)
- [PDF](https://arxiv.org/pdf/2304.09919)
- [arXiv 2304.09919](https://arxiv.org/abs/2304.09919)
- [OpenAlex](https://openalex.org/W4366732166)

**Authors (10)**

1. [Vesa Akerman](https://openalex.org/A5071402469) [ORCID](https://orcid.org/0000-0003-0612-2378) · [Semantic Scholar](https://www.semanticscholar.org/author/72235072) · h 2 · 52 citations · 11 works
2. [David Baines](https://openalex.org/A5101753048) [ORCID](https://orcid.org/0009-0004-5725-1748) · [Semantic Scholar](https://www.semanticscholar.org/author/144810481) · h 1 · 3 citations · 2 works
3. [Damien Daspit](https://openalex.org/A5047308668) [Semantic Scholar](https://www.semanticscholar.org/author/2375782) · h 3 · 36 citations · 6 works
4. [Ulf Hermjakob](https://openalex.org/A5044761050) [Semantic Scholar](https://www.semanticscholar.org/author/1791311) · h 23 · 3054 citations · 71 works
5. Taeho Jang [Semantic Scholar](https://www.semanticscholar.org/author/2175479735)
6. [Colin Leong](https://openalex.org/A5084687814) [Semantic Scholar](https://www.semanticscholar.org/author/89269402) · h 2 · 15 citations · 5 works
7. Michael Martin [Semantic Scholar](https://www.semanticscholar.org/author/2214850451)
8. [Joel Mathew](https://openalex.org/A5078404774) [Semantic Scholar](https://www.semanticscholar.org/author/2064861181) · h 4 · 64 citations · 20 works
9. [Jonathan Robie](https://openalex.org/A5024539295) [Semantic Scholar](https://www.semanticscholar.org/author/1783920) · h 9 · 966 citations · 40 works
10. [Marcus Schwarting](https://openalex.org/A5083396472) [ORCID](https://orcid.org/0000-0001-6817-7265) · [Semantic Scholar](https://www.semanticscholar.org/author/47549959) · h 11 · 891 citations · 47 works

Citations and author metrics from openalex (2026-10-18); authors with the same name are not always told apart.

## What it is about

The paper presents the eBible corpus: 1,009 Bible translations in 833 languages from 75 language families, gathered from eBible.org and aligned verse by verse. The authors normalise versification and encoding, which differ between translations, since they block simple use of the data. They benchmark neural translation (NLLB-based models) on paired-verse data with three protocols: a standard train, test and validation split, a randomised cross-validation over translation pairs (because the sets are small, no 80/10/10 split is used), and tasks that follow a realistic translation order, such as Gospels first, then epistles, then Pentateuch, and separate New and Old Testament tasks. Scores are BLEU, spBLEU and chrF3. For one Bible translation task the Austronesian and Trans-New Guinea families reach 35.1 and 31.6 BLEU. The paper also notes that the corpus files have mixed licences: per the paper, some are CC BY-SA, two are CC BY-NC, and most are permissive.

## Ideas

- I1. Check the translator's loaders against the paper's main data problem: one verse in one translation can span two in another. A verse-key file (versification) that all translations are mapped onto is the fix; test whether `tools/translate` handles the merged-verse cases before measuring anything.
- I2. Adopt the randomised cross-validation over translation pairs as the protocol for the author-delta decisions, which matches the cv rule of the project.
- I3. Test the delta learner on a growing set of books in the paper's translation order, to measure how much of an author's delta is learned from a partial translation. Measure, do not assume.

## For Mova

The relevant parts are the verse alignment and the benchmark protocols for `tools/translate` and `measure.rs`. The NMT models are not for Mova. Because the corpus mixes licences file by file, the data stays outside the build and only the tools and protocols are taken.

Topics: [Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/topics/machine-translation/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [arXiv 2304.09919](https://arxiv.org/abs/2304.09919)

```
@misc{akerman2023ebible,
  title = {{The eBible Corpus: Data and Model Benchmarks for Bible Translation for Low-Resource Languages}},
  author = {Akerman, Vesa and Baines, David and Daspit, Damien and Hermjakob, Ulf and Jang, Taeho and Leong, Colin and Martin, Michael and Mathew, Joel and Robie, Jonathan and Schwarting, Marcus},
  year = {2023},
  eprint = {2304.09919},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2304.09919}
}
```

- [← previous The Mechanical Bard: An Interpretable Machine Learning Approach to Shakespearean Sonnet Generation](https://md.mova.elamur.ai/free/en/docs/papers/agnew-2023-mechanical-bard-interpretable-machine-learning/)
- [next → Translation by Quasi Logical Form Transfer](https://md.mova.elamur.ai/free/en/docs/papers/alshawi-1991-translation-quasi-logical-form-transfer/)
- [↑ back to Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/mtrans/)

## References

25 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Orevaoghene Ahia, Julia Kreutzer, and Sara Hooker. 2021. The low-resource double bind: An empirical study of pruning for low-resource machine translation. arXiv preprint arXiv:2110.03036. [↗](https://arxiv.org/abs/2110.03036)
2. Benjamin Akera, Jonathan Mukiibi, Lydia Sanyu Naggayi, Claire Babirye, Isaac Owomugisha, Solomon Nsumba, Joyce Nakatumba-Nabende, Engineer Bainomugisha, Ernest Mwebaze, and John Quinn. 2022. Machine translation for African languages: Community creation of datasets and models in Uganda. [find](https://scholar.google.com/scholar?q=Benjamin+Akera%2C+Jonathan+Mukiibi%2C+Lydia+Sanyu+Naggayi%2C+Claire+Babirye%2C+Isaac+Owomugisha%2C+Solomon+Nsumba%2C+Joyce+Nakatumba%2DNabende%2C+Engineer+Bainomugisha%2C+Ernest+Mwebaze%2C+and+John+Quinn%2E+2022%2E+Machine+t)
3. David Baines, Joel Mathew, and Michael Martin. 2023a. The eBible corpus. [https://github.com/](https://github.com/) BibleNLP/ebible. [↗](https://github.com/)
4. David Baines, Joel Mathew, and Michael Martin. 2023b. eBible corpus NMT experiments. [https://github.com/BibleNLP/ebible-experiments](https://github.com/BibleNLP/ebible-experiments). [↗](https://github.com/BibleNLP/ebible-experiments)
5. Christos Christodouloupoulos and Mark Steedman. 2015. A massively parallel corpus: the Bible in 100 languages. Language resources and evaluation, 49:375–395. [find](https://scholar.google.com/scholar?q=Christos+Christodouloupoulos+and+Mark+Steedman%2E+2015%2E+A+massively+parallel+corpus%3A+the+Bible+in+100+languages%2E+Language+resources+and+evaluation%2C+49%3A375%E2%80%93395%2E)
6. Richard Oliver Collin. 2010. Ethnologue. Ethnopoli-tics, 9(3-4):425–432. [find](https://scholar.google.com/scholar?q=Richard+Oliver+Collin%2E+2010%2E+Ethnologue%2E+Ethnopoli%2Dtics%2C+9%283%2D4%29%3A425%E2%80%93432%2E)
7. Marta R Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672. [↗](https://arxiv.org/abs/2207.04672)
8. Damien Daspit. 2023. SIL-NLP. [https://github.com/sillsdev/silnlp](https://github.com/sillsdev/silnlp). [↗](https://github.com/sillsdev/silnlp)
9. Yonggang Deng and William Byrne. 2008. HMM word and phrase alignment for statistical machine translation. IEEE Transactions on Audio, Speech, and Language Processing, 16(3):494–507. [find](https://scholar.google.com/scholar?q=Yonggang+Deng+and+William+Byrne%2E+2008%2E+HMM+word+and+phrase+alignment+for+statistical+machine+translation%2E+IEEE+Transactions+on+Audio%2C+Speech%2C+and+Language+Processing%2C+16%283%29%3A494%E2%80%93507%2E)
10. Chris Dyer, Victor Chahuneau, and Noah A Smith. 2013. A simple, fast, and effective reparameterization of IBM model 2. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 644–648. [find](https://scholar.google.com/scholar?q=Chris+Dyer%2C+Victor+Chahuneau%2C+and+Noah+A+Smith%2E+2013%2E+A+simple%2C+fast%2C+and+effective+reparameterization+of+IBM+model+2%2E+In+Proceedings+of+the+2013+Conference+of+the+North+American+Chapter+of+the+Associ)
11. Naman Goyal, Cynthia Gao, Vishrav Chaudhary, Peng-Jen Chen, Guillaume Wenzek, Da Ju, Sanjana Krishnan, Marc’Aurelio Ranzato, Francisco Guzmán, and Angela Fan. 2022. The flores-101 evaluation benchmark for low-resource and multilingual machine translation. Transactions of the Association for Computational Linguistics, 10:522–538. [find](https://scholar.google.com/scholar?q=Naman+Goyal%2C+Cynthia+Gao%2C+Vishrav+Chaudhary%2C+Peng%2DJen+Chen%2C+Guillaume+Wenzek%2C+Da+Ju%2C+Sanjana+Krishnan%2C+Marc%E2%80%99Aurelio+Ranzato%2C+Francisco+Guzm%C3%A1n%2C+and+Angela+Fan%2E+2022%2E+The+flores%2D101+evaluation+benchmark)
12. Ulf Hermjakob. 2023. Wildebeest. [https://github.com/uhermjakob/wildebeest](https://github.com/uhermjakob/wildebeest). [↗](https://github.com/uhermjakob/wildebeest)
13. Katharina Kann, Abteen Ebrahimi, Manuel Mager, Arturo Oncevay, John E Ortega, Annette Rios, Angela Fan, Ximena Gutierrez-Vasques, Luis Chiruzzo, Gustavo A Giménez-Lugo, et al. 2022. Americasnli: Machine translation and natural language inference systems for Indigenous languages of the Americas. Frontiers in Artificial Intelligence, 5:266. [find](https://scholar.google.com/scholar?q=Katharina+Kann%2C+Abteen+Ebrahimi%2C+Manuel+Mager%2C+Arturo+Oncevay%2C+John+E+Ortega%2C+Annette+Rios%2C+Angela+Fan%2C+Ximena+Gutierrez%2DVasques%2C+Luis+Chiruzzo%2C+Gustavo+A+Gim%C3%A9nez%2DLugo%2C+et+al%2E+2022%2E+Americasnli%3A+Machi)
14. Jeff Klassen. 2022. Unified standard format markers. [https://github.com/ubsicap/usfm](https://github.com/ubsicap/usfm). [↗](https://github.com/ubsicap/usfm)
15. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M Rush. 2017. Opennmt: Open-source toolkit for neural machine translation. arXiv preprint arXiv:1701.02810. [↗](https://arxiv.org/abs/1701.02810)
16. Philipp Koehn. 2009. Statistical machine translation. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2E+2009%2E+Statistical+machine+translation%2E+Cambridge+University+Press%2E)
17. Sami Liedes. 2018. Machine translating the Bible into new languages. [https://github.com/sliedes/](https://github.com/sliedes/) fairseq-py. [↗](https://github.com/sliedes/)
18. Arya D McCarthy, Rachel Wicks, Dylan Lewis, Aaron Mueller, Winston Wu, Oliver Adams, Garrett Nicolai, Matt Post, and David Yarowsky. 2020. The Johns Hopkins University Bible corpus: 1600+ tongues for typological exploration. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 2884–2892. [find](https://scholar.google.com/scholar?q=Arya+D+McCarthy%2C+Rachel+Wicks%2C+Dylan+Lewis%2C+Aaron+Mueller%2C+Winston+Wu%2C+Oliver+Adams%2C+Garrett+Nicolai%2C+Matt+Post%2C+and+David+Yarowsky%2E+2020%2E+The+Johns+Hopkins+University+Bible+corpus%3A+1600%2B+tongues+for+)
19. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. arXiv preprint arXiv:1904.01038. [↗](https://arxiv.org/abs/1904.01038)
20. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+annual+meeting+of+the+Association+for+Comp) [our card](https://md.mova.elamur.ai/free/en/docs/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
21. Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebastian Ruder, Kyunghyun Cho, and Iryna Gurevych. 2020. Adapterhub: A framework for adapting transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 46–54. [find](https://scholar.google.com/scholar?q=Jonas+Pfeiffer%2C+Andreas+R%C3%BCckl%C3%A9%2C+Clifton+Poth%2C+Aishwarya+Kamath%2C+Ivan+Vulic%CC%81%2C+Sebastian+Ruder%2C+Kyunghyun+Cho%2C+and+Iryna+Gurevych%2E+2020%2E+Adapterhub%3A+A+framework+for+adapting+transformers%2E+In+Proceedings)
22. Maja Popović. 2015. chrf: character n-gram F-score for automatic MT evaluation. In Proceedings of the tenth workshop on statistical machine translation, pages 392–395. [find](https://scholar.google.com/scholar?q=Maja+Popovic%CC%81%2E+2015%2E+chrf%3A+character+n%2Dgram+F%2Dscore+for+automatic+MT+evaluation%2E+In+Proceedings+of+the+tenth+workshop+on+statistical+machine+translation%2C+pages+392%E2%80%93395%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/popovic-2015-chrf-character-n-gram-f/)
23. Philip Resnik, Mari Broman Olsen, and Mona Diab. 1999. The Bible as a parallel corpus: Annotating the ‘Book of 2000 Tongues’. Computers and the Humanities, 33:129–153. [find](https://scholar.google.com/scholar?q=Philip+Resnik%2C+Mari+Broman+Olsen%2C+and+Mona+Diab%2E+1999%2E+The+Bible+as+a+parallel+corpus%3A+Annotating+the+%E2%80%98Book+of+2000+Tongues%E2%80%99%2E+Computers+and+the+Humanities%2C+33%3A129%E2%80%93153%2E)
24. Winston Wu, Nidhi Vyas, and David Yarowsky. 2018. Creating a translation matrix of the Bible’s names across 591 languages. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). [find](https://scholar.google.com/scholar?q=Winston+Wu%2C+Nidhi+Vyas%2C+and+David+Yarowsky%2E+2018%2E+Creating+a+translation+matrix+of+the+Bible%E2%80%99s+names+across+591+languages%2E+In+Proceedings+of+the+Eleventh+International+Conference+on+Language+Resources) [our card](https://md.mova.elamur.ai/free/en/docs/papers/wu-2018-creating-translation-matrix-bible-s/)
25. Jinhua Zhu, Yingce Xia, Lijun Wu, Di He, Tao Qin, Wengang Zhou, Houqiang Li, and Tie-Yan Liu. 2020. Incorporating bert into neural machine translation. arXiv preprint arXiv:2002.06823. [↗](https://arxiv.org/abs/2002.06823)