[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/aclbulk-2011-scheible-gold-standard-corpus-early-modern/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / A Gold Standard Corpus of Early Modern German

# A Gold Standard Corpus of Early Modern German

**Authors:** Silke Scheible, Richard J. Whitt, Martin Durrell and Paul Bennett · **Year:** 2011 · **Venue:** Proceedings of the 5th Linguistic Annotation Workshop **Link:** [aclanthology.org/W11-0415](https://aclanthology.org/W11-0415/) · **Licence:** [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/) — summary only

[**31** citations](https://openalex.org/works?filter=cites:W2143092851) openalex-snapshot · 2026-10-14[**17** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/aclbulk-2011-scheible-gold-standard-corpus-early-modern/) of the 140673 papers whose bibliographies we read[Proceedings of the 5th Linguistic Annotation Workshop](https://aclanthology.org/volumes/W11-04/)[licence](https://creativecommons.org/licenses/by-nc-sa/3.0/)

- [Publisher's page](https://aclanthology.org/W11-0415/)
- [PDF](https://aclanthology.org/W11-0415.pdf)
- [OpenAlex](https://openalex.org/W2143092851)

**Authors (4)**

1. [Silke Scheible](https://openalex.org/A5001633683) h 7 · 256 citations · 25 works
2. [Richard J. Whitt](https://openalex.org/A5020302669) [ORCID](https://orcid.org/0000-0002-5249-671X) · h 14 · 861 citations · 61 works
3. [Martin Durrell](https://openalex.org/A5112741841) h 13 · 725 citations · 148 works
4. [Paul Bennett](https://openalex.org/A5109801636) h 21 · 1367 citations · 165 works

Citations and author metrics from openalex-snapshot (2026-10-14); authors with the same name are not always told apart.

## What it is about

The paper describes a hand-annotated gold standard sample of Early Modern German, taken from the GerManC corpus (1650–1800). It has 57,845 tokens from the North German region, one sample per genre and per 50-year period. Each token has a POS tag, a lemma and a normalised spelling. Two annotators worked on it. Agreement is 96.9% on normalised forms, 95.5% on lemmas and 91.6% on POS tags. POS agreement is lower in the earliest period (89.3%) than in the latest (93.1%). The authors plan to use the corpus to test modern tools on historical data, including the normalised layer.

## Ideas

- I1. `en/src/morph.rs`: keep the normalised spelling as its own layer next to the lemma, as this corpus does, so that the `archaic` mapping can be tested apart from the lemma step. Measure the lemma accuracy on the test split of an early-English set with and without this layer.
- I2. `en tag-eval`: report tagging accuracy per period and per genre, not only overall, as the paper's period breakdown does. Use the same split on tools/gutenberg lessons from different centuries, and note the date of each run.
- I3. Agreement ceiling: double-annotate about 500 archaic tokens and compute agreement on POS and lemma before tuning `en morph-eval` on them. This gives a ceiling for each historical test set.

## For Mova

The chapters are [09. Early Modern and archaic forms](https://md.mova.elamur.ai/free/en/docs/languages/en/09-early-modern-and-archaic/) and [06. Spelling rules: inflection, derivation, compounds and national standards](https://md.mova.elamur.ai/free/en/docs/languages/en/06-spelling-rules/). The code is `en/src/morph.rs` and the `en tag-eval` command. The corpus is German and NC, so no number from it can be used as an English figure, and it is for tests only.

Topics: [Part-of-speech tagging](https://md.mova.elamur.ai/free/en/docs/papers/topics/part-of-speech-tagging/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work.

```
@inproceedings{scheible2011gold,
  title = {{A Gold Standard Corpus of Early Modern German}},
  author = {Scheible, Silke and Whitt, Richard J. and Durrell, Martin and Bennett, Paul},
  booktitle = {Proceedings of the 5th Linguistic Annotation Workshop},
  year = {2011},
  url = {https://aclanthology.org/W11-0415/}
}
```

- [← previous Domain-Dependent Detection of Light Verb Constructions](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2011-nagy-domain-dependent-detection-light-verb/)
- [next → Evaluating Dependency Parsing: Robust and Heuristics-Free Cross-Annotation Evaluation](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2011-tsarfaty-evaluating-dependency-parsing-robust-heuristics/)
- [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/)

## References

14 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alistair Baron and Paul Rayson. 2008. VARD 2: A tool for dealing with spelling variation in historical corpora. Proceedings of the Postgraduate Conference in Corpus Linguistics, Birmingham, UK. [find](https://scholar.google.com/scholar?q=Alistair+Baron+and+Paul+Rayson%2E+2008%2E+VARD+2%3A+A+tool+for+dealing+with+spelling+variation+in+historical+corpora%2E+Proceedings+of+the+Postgraduate+Conference+in+Corpus+Linguistics%2C+Birmingham%2C+UK%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/enprag-2008-baron-vard2-tool-dealing-spelling-variation/)
2. Douglas Biber and Edward Finegan. 1989. Drift and the evolution of English style: a history of three genres. Language 65. 487-517. [↗](https://doi.org/10.2307/415220)
3. Torsten Brants. 2000a. Inter-annotator agreement for a German newspaper corpus. Second International Conference on Language Resources and Evaluation (LREC 2000), Athens, Greece. [find](https://scholar.google.com/scholar?q=Torsten+Brants%2E+2000a%2E+Inter%2Dannotator+agreement+for+a+German+newspaper+corpus%2E+Second+International+Conference+on+Language+Resources+and+Evaluation+%28LREC+2000%29%2C+Athens%2C+Greece%2E)
4. Torsten Brants. 2000b. TnT – a statistical part-of-speech tagger. Proceedings of the 6th Applied NLP Conference, ANLP-2000, Seattle, WA. [↗](https://doi.org/10.48550/arxiv.cs/0003055) [our card](https://md.mova.elamur.ai/free/en/docs/papers/brants-2000-tnt-statistical-part-speech-tagger/)
5. Hamish Cunningham, Diana Maynard, Kalina Bontcheva, and Valentin Tablan. 2002. GATE: A framework and graphical development environment for robust NLP tools and applications. Proceedings of the 40th Anniversary Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hamish+Cunningham%2C+Diana+Maynard%2C+Kalina+Bontcheva%2C+and+Valentin+Tablan%2E+2002%2E+GATE%3A+A+framework+and+graphical+development+environment+for+robust+NLP+tools+and+applications%2E+Proceedings+of+the+40th+An)
6. Stefanie Dipper. 2010. POS-Tagging of historical language data: First experiments in semantic approaches in Natural Language Processing. Proceedings of the 10th Conference on Natural Language Processing (KONVENS-10). Saarbrücken, Germany. 117-121. [find](https://scholar.google.com/scholar?q=Stefanie+Dipper%2E+2010%2E+POS%2DTagging+of+historical+language+data%3A+First+experiments+in+semantic+approaches+in+Natural+Language+Processing%2E+Proceedings+of+the+10th+Conference+on+Natural+Language+Processi)
7. Andrea Ernst-Gerlach and Norbert Fuhr. 2006. Generating search term variants for text collections with historic spellings. Proceedings of the 28th European Conference on Information Retrieval Research (ECIR 2006), London, UK. [find](https://scholar.google.com/scholar?q=Andrea+Ernst%2DGerlach+and+Norbert+Fuhr%2E+2006%2E+Generating+search+term+variants+for+text+collections+with+historic+spellings%2E+Proceedings+of+the+28th+European+Conference+on+Information+Retrieval+Research)
8. Vera Fasshauer. 2011. [http://www.indogermanistik.uni-jena.de/index.php?auswahl=184](http://www.indogermanistik.uni-jena.de/index.php?auswahl=184) Accessed 30/03/2011. [↗](http://www.indogermanistik.uni-jena.de/index.php?auswahl=184)
9. Bryan Jurish. 2010. Comparing canonicalizations of historical German text. Proceedings of the 11th Meeting of the ACL Special Interest Group on Computational Morphology and Phonology (SIGMORPHON), Uppsala, Sweden. 72-77. [find](https://scholar.google.com/scholar?q=Bryan+Jurish%2E+2010%2E+Comparing+canonicalizations+of+historical+German+text%2E+Proceedings+of+the+11th+Meeting+of+the+ACL+Special+Interest+Group+on+Computational+Morphology+and+Phonology+%28SIGMORPHON%29%2C+Upp)
10. Thomas Pilz and Wolfram Luther. 2009. Automated support for evidence retrieval in documents with non-standard orthography. The Fruits of Empirical Linguistics. Sam Featherston and Susanne Winkler (eds.). 211–228. [find](https://scholar.google.com/scholar?q=Thomas+Pilz+and+Wolfram+Luther%2E+2009%2E+Automated+support+for+evidence+retrieval+in+documents+with+non%2Dstandard+orthography%2E+The+Fruits+of+Empirical+Linguistics%2E+Sam+Featherston+and+Susanne+Winkler+%28eds)
11. Paul Rayson, Dawn Archer, Alistair Baron, Jonathan Culpeper, and Nicholas Smith. 2007. Tagging the Bard: Evaluating the accuracy of a modern POS tagger on Early Modern English corpora. Proceedings of the Corpus Linguistics Conference (CL2007), University of Birmingham, UK. [find](https://scholar.google.com/scholar?q=Paul+Rayson%2C+Dawn+Archer%2C+Alistair+Baron%2C+Jonathan+Culpeper%2C+and+Nicholas+Smith%2E+2007%2E+Tagging+the+Bard%3A+Evaluating+the+accuracy+of+a+modern+POS+tagger+on+Early+Modern+English+corpora%2E+Proceedings+of+)
12. Silke Scheible, Richard J. Whitt, Martin Durrell, and Paul Bennett. 2010. Annotating a Historical Corpus of German: A Case Study. Proceedings of the LREC 2010 Workshop on Language Resources and Language Technology Standards, Valletta, Malta. [find](https://scholar.google.com/scholar?q=Silke+Scheible%2C+Richard+J%2E+Whitt%2C+Martin+Durrell%2C+and+Paul+Bennett%2E+2010%2E+Annotating+a+Historical+Corpus+of+German%3A+A+Case+Study%2E+Proceedings+of+the+LREC+2010+Workshop+on+Language+Resources+and+Langua)
13. Anne Schiller, Simone Teufel, Christine Stöckert, and Christine Thielen. 1999. Guidelines für das Tagging deutscher Textcorpora mit STTS. Technical Report. Institut für maschinelle Sprachverarbeitung, Stuttgart. [find](https://scholar.google.com/scholar?q=Anne+Schiller%2C+Simone+Teufel%2C+Christine+Sto%CC%88ckert%2C+and+Christine+Thielen%2E+1999%2E+Guidelines+fu%CC%88r+das+Tagging+deutscher+Textcorpora+mit+STTS%2E+Technical+Report%2E+Institut+fu%CC%88r+maschinelle+Sprachverarbeitu)
14. Helmut Schmid. 1994. Probabilistic part-of-speech tagging using decision trees. International Conference on New Methods in Language Processing, Manchester, UK. 44–49. [find](https://scholar.google.com/scholar?q=Helmut+Schmid%2E+1994%2E+Probabilistic+part%2Dof%2Dspeech+tagging+using+decision+trees%2E+International+Conference+on+New+Methods+in+Language+Processing%2C+Manchester%2C+UK%2E+44%E2%80%9349%2E)