[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/bollmann-2011-rule-based-normalization-historical-texts/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/mtrans/) / Rule-Based Normalization of Historical Texts

# Rule-Based Normalization of Historical Texts

**Authors:** [Marcel Bollmann](https://md.mova.elamur.ai/free/en/docs/papers/authors/marcel-bollmann/), [Florian Petran](https://md.mova.elamur.ai/free/en/docs/papers/authors/florian-petran/), [Stefanie Dipper](https://md.mova.elamur.ai/free/en/docs/papers/authors/stefanie-dipper/) · **Year:** 2011 · **Venue:** Proceedings of the Workshop on Language Technologies for Digital Humanities and Cultural Heritage **Link:** [aclanthology.org/W11-4106](https://aclanthology.org/W11-4106/) · **Licence:** [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/) — summary only

[**34** citations](https://openalex.org/works?filter=cites:W779297117) openalex · 2026-10-19[**18** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/bollmann-2011-rule-based-normalization-historical-texts/) of the 140673 papers whose bibliographies we read[Proceedings of the Workshop on Language Technologies for Digital Humanities and Cultural Heritage](https://aclanthology.org/volumes/W11-41/)[licence](https://creativecommons.org/licenses/by-nc-sa/3.0/)

- [Publisher's page](https://aclanthology.org/W11-4106/)
- [PDF](https://aclanthology.org/W11-4106.pdf)
- [OpenAlex](https://openalex.org/W779297117)

**Authors (3)**

1. [Marcel Bollmann](https://openalex.org/A5005438563) [ORCID](https://orcid.org/0000-0003-2598-8150) · [Semantic Scholar](https://www.semanticscholar.org/author/34887843) · h 11 · 337 citations · 42 works
2. [Florian Petran](https://openalex.org/A5031866050) [Semantic Scholar](https://www.semanticscholar.org/author/2964686) · h 5 · 102 citations · 13 works
3. [Stefanie Dipper](https://openalex.org/A5063444213) [ORCID](https://orcid.org/0000-0003-4357-9078) · [Semantic Scholar](https://www.semanticscholar.org/author/1889046) · h 18 · 1113 citations · 146 works

Citations and author metrics from openalex (2026-10-19); authors with the same name are not always told apart.

## What it is about

The paper normalizes Early New High German spelling to modern forms using only unsupervised learning and freely available resources. The training material is two aligned versions of the Luther Bible, which give word pairs that differ in spelling. From these the method derives context-aware rewrite rules over character sequences, including identity rules that leave a character unchanged, and weights each rule by its frequency. To normalize a word, the rules are applied to each character and the candidates are checked against a modern lexicon. The evaluation finds that the rules reach 83 to 91 percent exact matches, against 65 percent for leaving the text unchanged. The main open issue is the multi-path problem: one historical word can be reached from the source by several rule sequences, and the method does not yet choose the right one reliably.

## Ideas

- I1. Derive word-level rewrite rules from aligned versions of the Bible, for example KJV against WEB at verse level. A rule is a change with its context and a frequency weight, and identity rules keep unchanged words. This gives the author delta as an explicit rule table. Measure exact matches on dev verses against copying the standard.
- I2. Handle the multi-path problem with the paper's own suggestion: keep the derivation with the fewest rules. Count ties in BTreeMap order, so the choice is deterministic, and report how often ties occur. The same ambiguity appears in Hebrew word alignment.
- I3. Frequency-weighted identity rules are a count-based form of the standard-by-default rule. A delta then needs a rule with real counts behind it, which matches the threshold on overrides already used in `tools/translate`.
- I4. Use the rule count as a size measure of a delta. A small rule table for an author that is close to the standard is a sign that the author delta is well founded; a large one points to noise.

## More ideas (a second reading)

- I5. `en/src/morph.rs`: derive rewrite rules from aligned archaic and modern forms in the tools/gutenberg lessons (for example hast to has, doth to does, -eth to -s), rank them by frequency, and measure the exact-match rate on held-out lessons against the `archaic` function.
- I6. Keep both rankers: a best-rule ranker for unseen words and a best-probability ranker for seen words. The 3% of words where the two disagree show where a second method adds something. Test the combination with `en folds`.
- I7. Clitic contractions: the paper's example soltu for sollst du is a rule family of verb-plus-pronoun forms. Count how often the thou-forms (art, dost, hast) in tools/gutenberg lessons are fixed by the `archaic` function, and add rules only for the misses.

## For Mova

The code is `tools/translate` (the delta side, and `profile.rs` for the standard). The method is rule-based and count-based, so it fits Mova's design directly. It addresses open problem 6 by deriving the delta from the data, and open problem 2 through the multi-path choice.

Topics: [Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/topics/machine-translation/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work.

```
@inproceedings{bollmann2011rule,
  title = {{Rule-Based Normalization of Historical Texts}},
  author = {Bollmann, Marcel and Petran, Florian and Dipper, Stefanie},
  booktitle = {Proceedings of the Workshop on Language Technologies for Digital Humanities and Cultural Heritage},
  year = {2011},
  url = {https://aclanthology.org/W11-4106/}
}
```

- [← previous Phrase-Based and Deep Syntactic English-to-Czech Statistical Machine Translation](https://md.mova.elamur.ai/free/en/docs/papers/bojar-2008-phrase-based-deep-syntactic-english/)
- [next → A Large-Scale Comparison of Historical Text Normalization Systems](https://md.mova.elamur.ai/free/en/docs/papers/bollmann-2019-large-scale-comparison-historical-text/)
- [↑ back to Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/mtrans/)

## References

11 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alistair Baron, Paul Rayson, and Dawn Archer. 2009. Automatic standardization of spelling for historical text mining. In Proceedings of Digital Humanities. [find](https://scholar.google.com/scholar?q=Alistair+Baron%2C+Paul+Rayson%2C+and+Dawn+Archer%2E+2009%2E+Automatic+standardization+of+spelling+for+historical+text+mining%2E+In+Proceedings+of+Digital+Humanities%2E)
2. Karin Beijering, Charlotte Gooskens, and Wilbert Heeringa. 2008. Predicting intelligibility and perceived linguistic distances by means of the Levenshtein algorithm. Linguistics in the Netherlands, pages 13–24. [find](https://scholar.google.com/scholar?q=Karin+Beijering%2C+Charlotte+Gooskens%2C+and+Wilbert+Heeringa%2E+2008%2E+Predicting+intelligibility+and+perceived+linguistic+distances+by+means+of+the+Levenshtein+algorithm%2E+Linguistics+in+the+Netherlands%2C+pa)
3. Werner Besch. 2000. Die Rolle Luthers für die deutsche Sprachgeschichte. In Werner Besch et al., editor, Sprachgeschichte. Ein Handbuch zur Geschichte der deutschen Sprache und ihrer Erforschung, pages 1713–1745. de Gruyter, 2nd edition. [find](https://scholar.google.com/scholar?q=Werner+Besch%2E+2000%2E+Die+Rolle+Luthers+fu%CC%88r+die+deutsche+Sprachgeschichte%2E+In+Werner+Besch+et+al%2E%2C+editor%2C+Sprachgeschichte%2E+Ein+Handbuch+zur+Geschichte+der+deutschen+Sprache+und+ihrer+Erforschung%2C+pag)
4. Fabienne Braune and Alexander Fraser. 2010. Improved unsupervised sentence alignment for symmetrical and asymmetrical parallel corpora. In Proceedings of COLING, Poster Volume, pages 81–89. [find](https://scholar.google.com/scholar?q=Fabienne+Braune+and+Alexander+Fraser%2E+2010%2E+Improved+unsupervised+sentence+alignment+for+symmetrical+and+asymmetrical+parallel+corpora%2E+In+Proceedings+of+COLING%2C+Poster+Volume%2C+pages+81%E2%80%9389%2E)
5. Andrea Ernst-Gerlach and Norbert Fuhr. 2006. Generating search term variants for text collections with historic spellings. In Proceedings of the 28th European Conference on Information Retrieval Research (ECIR 2006). Springer. [find](https://scholar.google.com/scholar?q=Andrea+Ernst%2DGerlach+and+Norbert+Fuhr%2E+2006%2E+Generating+search+term+variants+for+text+collections+with+historic+spellings%2E+In+Proceedings+of+the+28th+European+Conference+on+Information+Retrieval+Resea)
6. Andreas W. Hauser and Klaus U. Schulz. 2007. Unsupervised learning of edit distance weights for retrieving historical spelling variations. In Proceedings of the First Workshop on Finite-State Techniques and Approximate Search, pages 1–6. [find](https://scholar.google.com/scholar?q=Andreas+W%2E+Hauser+and+Klaus+U%2E+Schulz%2E+2007%2E+Unsupervised+learning+of+edit+distance+weights+for+retrieving+historical+spelling+variations%2E+In+Proceedings+of+the+First+Workshop+on+Finite%2DState+Techniqu)
7. Bryan Jurish. 2010. More than words: Using token context to improve canonicalization of historical German. Journal for Language Technology and Computational Linguistics, 25(1):23–39. [find](https://scholar.google.com/scholar?q=Bryan+Jurish%2E+2010%2E+More+than+words%3A+Using+token+context+to+improve+canonicalization+of+historical+German%2E+Journal+for+Language+Technology+and+Computational+Linguistics%2C+25%281%29%3A23%E2%80%9339%2E)
8. Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In Proceedings of MT Summit. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2E+2005%2E+Europarl%3A+A+parallel+corpus+for+statistical+machine+translation%2E+In+Proceedings+of+MT+Summit%2E)
9. Franz Josef Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19–51. [find](https://scholar.google.com/scholar?q=Franz+Josef+Och+and+Hermann+Ney%2E+2003%2E+A+systematic+comparison+of+various+statistical+alignment+models%2E+Computational+Linguistics%2C+29%281%29%3A19%E2%80%9351%2E)
10. Jan Strunk. 2003. Information retrieval for languages that lack a fixed orthography. Seminar Paper, Stanford University. [http://www.linguistics](http://www.linguistics). rub.de/˜strunk/LSreport.pdf. [↗](http://www.linguistics)
11. Norbert Richard Wolf. 2000. Handschrift und Druck. In Werner Besch et al., editor, Sprachgeschichte. Ein Handbuch zur Geschichte der deutschen Sprache und ihrer Erforschung, pages 1705–1713. de Gruyter, 2nd edition. [find](https://scholar.google.com/scholar?q=Norbert+Richard+Wolf%2E+2000%2E+Handschrift+und+Druck%2E+In+Werner+Besch+et+al%2E%2C+editor%2C+Sprachgeschichte%2E+Ein+Handbuch+zur+Geschichte+der+deutschen+Sprache+und+ihrer+Erforschung%2C+pages+1705%E2%80%931713%2E+de+Gruyte)