[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/hbo-2019-joode-problem-oriented-corpus-annotation-hebrew/) [home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [Ancient Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/hbo/) / Problem-oriented Corpus Annotation and the Hebrew Bible # Problem-oriented Corpus Annotation and the Hebrew Bible **Authors:** Johan de Joode · **Year:** 2019 · **Venue:** HIPHIL Novum 5/2, 6– **Link:** [tidsskrift.dk/hiphilnovum/article/view/142730](https://tidsskrift.dk/hiphilnovum/article/view/142730) · **Licence:** copyrighted (free to read, no CC licence) — summary only [**0** citations](https://openalex.org/works?filter=cites:W4391938569) openalex · 2026-10-18[HIPHIL Novum 5](https://openalex.org/S5406958382)[licence](https://tidsskrift.dk/hiphilnovum/about) - [Publisher's page](https://doi.org/10.7146/hn.v5i2.142730) - [PDF](https://tidsskrift.dk/hiphilnovum/article/download/142730/186434) - [DOI 10.7146/hn.v5i2.142730](https://doi.org/10.7146/hn.v5i2.142730) - [OpenAlex](https://openalex.org/W4391938569) **Authors (1)** 1. [Johan de Joode](https://openalex.org/A5023540299) [ORCID](https://orcid.org/0000-0001-7209-880X) · h 5 · 2071 citations · 34 works Citations and author metrics from openalex (2026-10-18); authors with the same name are not always told apart. ## What it is about A position paper arguing that biblical scholars should add their own research-specific annotations to open datasets such as ETCBC ("problem-oriented annotation", de Haan 1984), instead of free-text notes in closed tools. From the author's experience annotating conceptual metaphors in Job (Biblia Metaphorica) and from Fort et al. (2012) on what makes annotation hard, he recommends: start from a query, not from running text, so units are pre-segmented; annotate in several passes, one word class or feature at a time; use small fixed tagsets, binary where possible, with one label for "needs more reflection"; keep annotations modular, stand-off and keyed to stable identifiers, in a format humans and machines can read; and measure quality by agreement (proportion agreeing, kappa). He also notes that no exhaustive Hebrew Bible dataset was then free for commercial use. ## Ideas - I1. When a teacher model or a person labels Hebrew phenomena for Mova (pragmatic acts, metaphor, animacy), give it pre-segmented units from a tree query and a closed, small label set with an explicit "unsure" label, one feature per pass. - I2. Keep such labels stand-off, keyed by `sent_id` + token id, never edited into the CoNLL-U, so they survive reconversion of the trees. - I3. A two-pass metaphor finder for the Torah: first label noun animacy (binary) in the dictionary, then flag clauses whose inanimate subject governs a verb that usually takes an animate subject. ## For Mova I1 and I2 are conventions for `macula prag`'s teacher comparison and for the parse database ([13. Biblical Hebrew in Mova: the tools](https://md.mova.elamur.ai/free/en/docs/languages/hbo/13-mova-tools/) §8, §10). I3 needs an animacy column in `macula dict` (`10-lexicon.md`, semantic fields) and a count over the trees; its output is a candidate list, not a claim. ## Cite The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.7146/hn.v5i2.142730](https://doi.org/10.7146/hn.v5i2.142730) ``` @inproceedings{joode2019problem, title = {{Problem-oriented Corpus Annotation and the Hebrew Bible}}, author = {Joode, Johan de}, booktitle = {HIPHIL Novum 5}, year = {2019}, doi = {10.7146/hn.v5i2.142730}, url = {https://doi.org/10.7146/hn.v5i2.142730} } ``` - [← previous Tyndale STEPBible Data development for machine analysis and computational linguistics](https://md.mova.elamur.ai/free/en/docs/papers/hbo-2019-instone-brewer-tyndale-stepbible-data-development-machine/) - [next → Is Biblical Hebrew a non-configurational language? Reconsidering the evidence from discontinuous phrases](https://md.mova.elamur.ai/free/en/docs/papers/hbo-2019-kaajan-biblical-hebrew-non-configurational-language/) - [↑ back to Ancient Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/hbo/) ## References 12 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Bayerl, Petra Saskia, and Karsten Ingmar Paul. (2011) “What Determines Inter-Coder Agreement in Manual Annotations? A Meta-Analytic Investigation” Computational Linguistics 37:4, 699–725. Available online from http://www.mitpressjournals.org/doi/10.1162/COLI_a_00074 [↗](https://doi.org/10.1162/coli_a_00074) 2. Carletta, J. (1996) Assessing Agreement on Classification Tasks: The Kappa Statistic. Computational Linguistics 22:2. Available online from [http://homepages.inf.ed.ac.uk/jeanc/squib.pdf](http://homepages.inf.ed.ac.uk/jeanc/squib.pdf) [↗](http://homepages.inf.ed.ac.uk/jeanc/squib.pdf) 3. de Haan P. (1984) “Problem-oriented tagging of English corpus data.” In Corpus linguistics, J. Aarts and W. Meijs (eds.). Amsterdam: Rodopi. [find](https://scholar.google.com/scholar?q=de+Haan+P%2E+%281984%29+%E2%80%9CProblem%2Doriented+tagging+of+English+corpus+data%2E%E2%80%9D+In+Corpus+linguistics%2C+J%2E+Aarts+and+W%2E+Meijs+%28eds%2E%29%2E+Amsterdam%3A+Rodopi%2E) 4. Fort, Karën, Adeline Nazarenko, and Sophie Rosset. (2012) “Modeling the Complexity of Manual Annotation Tasks: A Grid of Analysis.” Proc. COLING 2012 Tech. Pap. Pages 895–910, COLING 2012, Mumbai, December 2012. [find](https://scholar.google.com/scholar?q=Fort%2C+Kar%C3%ABn%2C+Adeline+Nazarenko%2C+and+Sophie+Rosset%2E+%282012%29+%E2%80%9CModeling+the+Complexity+of+Manual+Annotation+Tasks%3A+A+Grid+of+Analysis%2E%E2%80%9D+Proc%2E+COLING+2012+Tech%2E+Pap%2E+Pages+895%E2%80%93910%2C+COLING+2012%2C+Mumbai%2C+Dec) 5. Leech, G. (2005) “Adding Linguistic Annotation” in Developing Linguistic Corpora: a Guide to Good Practice. M. Wynne (ed.). Oxford: Oxbow Books: 17-29. Available online from [http://ota.ox.ac.uk/docu-ments/creating/dlc/](http://ota.ox.ac.uk/docu-ments/creating/dlc/) [↗](http://ota.ox.ac.uk/docu-ments/creating/dlc/) 6. Mahlberg M., Stockwell P., de Joode J., Smith C., O’Donnell M. (2016). “CLiC Dickens: Novel Uses of Concordances for the Integration of Corpus Stylistics and Cognitive Poetics.” Corpora, 11 (3), 433-463. [find](https://scholar.google.com/scholar?q=Mahlberg+M%2E%2C+Stockwell+P%2E%2C+de+Joode+J%2E%2C+Smith+C%2E%2C+O%E2%80%99Donnell+M%2E+%282016%29%2E+%E2%80%9CCLiC+Dickens%3A+Novel+Uses+of+Concordances+for+the+Integration+of+Corpus+Stylistics+and+Cognitive+Poetics%2E%E2%80%9D+Corpora%2C+11+%283%29%2C+433%2D4) 7. Manning, Christopher D., Prabhakar Ragahvan, and Hinrich Schutze. (2009) “An Introduction to Information Retrieval.” Information Retrieval. Cambridge: Cambridge University Press. Available online from [http://nlp.stanford.edu/IR-book/pdf/irbookonlinereading.pdf](http://nlp.stanford.edu/IR-book/pdf/irbookonlinereading.pdf) 6 Amazon’s Mechanical Turk can make this even more scalable. McEnery, Tony, and Andrew. [↗](http://nlp.stanford.edu/IR-book/pdf/irbookonlinereading.pdf) 8. Hardie. (2012) Corpus Linguistics Method, Theory and Practice. Cambridge: Cambridge University Press. [find](https://scholar.google.com/scholar?q=Hardie%2E+%282012%29+Corpus+Linguistics+Method%2C+Theory+and+Practice%2E+Cambridge%3A+Cambridge+University+Press%2E) 9. McEnery, Tony, and Andrew Wilson. (2011) Corpus Linguistics: An Introduction. 2nd edition. Edinburgh: Edinburgh University Press. [find](https://scholar.google.com/scholar?q=McEnery%2C+Tony%2C+and+Andrew+Wilson%2E+%282011%29+Corpus+Linguistics%3A+An+Introduction%2E+2nd+edition%2E+Edinburgh%3A+Edinburgh+University+Press%2E) 10. McIntyre, Dan. (2007) “Trusting the Text: Corpus Linguistics and Stylistics.” International Journal of Corpus Linguistics 12:4, 563–75. [find](https://scholar.google.com/scholar?q=McIntyre%2C+Dan%2E+%282007%29+%E2%80%9CTrusting+the+Text%3A+Corpus+Linguistics+and+Stylistics%2E%E2%80%9D+International+Journal+of+Corpus+Linguistics+12%3A4%2C+563%E2%80%9375%2E) 11. Pragglejaz Group. (2007). “MIP: A Method for Identifying Metaphorically Used Words in Discourse.” Metaphor and Symbol 22:1, 1-39. [find](https://scholar.google.com/scholar?q=Pragglejaz+Group%2E+%282007%29%2E+%E2%80%9CMIP%3A+A+Method+for+Identifying+Metaphorically+Used+Words+in+Discourse%2E%E2%80%9D+Metaphor+and+Symbol+22%3A1%2C+1%2D39%2E) 12. Sinclair, John. (2004) Trust the Text: Language, Corpus and Discourse. London: Routledge, 2004. [find](https://scholar.google.com/scholar?q=Sinclair%2C+John%2E+%282004%29+Trust+the+Text%3A+Language%2C+Corpus+and+Discourse%2E+London%3A+Routledge%2C+2004%2E)