[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/cl-2006-kiss-unsupervised-multilingual-sentence-boundary-detection/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / Unsupervised Multilingual Sentence Boundary Detection

# Unsupervised Multilingual Sentence Boundary Detection

**Authors:** [Tibor Kiss](https://md.mova.elamur.ai/free/en/docs/papers/authors/tibor-kiss/), Jan Strunk · **Year:** 2006 · **Venue:** Computational Linguistics **Link:** [aclanthology.org/J06-4003](https://aclanthology.org/J06-4003/) · **Licence:** [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/) — summary only

[**414** citations](https://openalex.org/works?filter=cites:W2121764873) openalex-snapshot · 2026-10-14[**109** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/cl-2006-kiss-unsupervised-multilingual-sentence-boundary-detection/) of the 140673 papers whose bibliographies we read[Computational Linguistics](https://aclanthology.org/volumes/J06-4/)[licence](https://creativecommons.org/licenses/by-nc-sa/3.0/)

- [Publisher's page](https://aclanthology.org/J06-4003/)
- [PDF](https://aclanthology.org/J06-4003.pdf)
- [DOI 10.1162/coli.2006.32.4.485](https://doi.org/10.1162/coli.2006.32.4.485)
- [OpenAlex](https://openalex.org/W2121764873)

**Authors (2)**

1. [Tibor Kiss](https://openalex.org/A5080795652) [ORCID](https://orcid.org/0000-0003-1857-3686) · [Semantic Scholar](https://www.semanticscholar.org/author/144372201) · h 12 · 1139 citations · 64 works
2. [Jan Strunk](https://openalex.org/A5066436233) [ORCID](https://orcid.org/0000-0001-8546-1778) · [Semantic Scholar](https://www.semanticscholar.org/author/35149376) · h 10 · 718 citations · 32 works

Citations and author metrics from openalex-snapshot (2026-10-14); authors with the same name are not always told apart.

## What it is about

The authors build a sentence boundary detector that needs no annotated data and no language-specific resources. Most ambiguous periods come from abbreviations, so the system first finds abbreviations from an unannotated corpus. Three criteria decide this, and all three use only the candidate type, not its context: a truncated word with a final period is a tight collocation, abbreviations are usually short, and some contain internal periods. The same collocation evidence also helps with initials and ordinal numbers. The system was developed on English and tested on eleven languages with newspaper corpora. Without any change to the system, the average error rate for sentence boundaries is 1.26%, ranging from 0.35% on German to 2.12% on Estonian, and the average error rate for abbreviation detection is 0.80%.

## Ideas

- I1. `en/src/tok.rs`: an abbreviation detector that scores each token ending in a period by its collocation with the period, its length and internal periods, learned from an unannotated English corpus, with a fixed threshold. Measure the sentence-boundary error rate on EWT test against the current rules, and the abbreviation error rate on the same tokens.
- I2. Use the paper's separate measures: report the error of abbreviation detection and of boundary detection, so a bad abbreviation list is visible in the number, not only in the final score.
- I3. Learn the initials and ordinal-number decisions with the same collocation counts, and check on EWT test whether the count of "A." and "1st." cases goes down, since the paper shows the gain only for these subtasks.

## For Mova

The code is `en/src/tok.rs`, with the metric from `en tok-eval`. The chapter is [07. Punctuation, capital letters, abbreviations and numbers](https://md.mova.elamur.ai/free/en/docs/languages/en/07-punctuation-capitals-numbers/). The collocation statistics come from a corpus, and the paper's own corpora are news in other languages, so the numbers are not Mova's. Use the English part only as a check. Measure first on EWT test, with a two-fold check (`en folds`).

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.1162/coli.2006.32.4.485](https://doi.org/10.1162/coli.2006.32.4.485)

```
@article{kiss2006unsupervised,
  title = {{Unsupervised Multilingual Sentence Boundary Detection}},
  author = {Kiss, Tibor and Strunk, Jan},
  journal = {Computational Linguistics},
  year = {2006},
  doi = {10.1162/coli.2006.32.4.485},
  url = {https://doi.org/10.1162/coli.2006.32.4.485}
}
```

- [← previous Head-Driven Statistical Models for Natural Language Parsing](https://md.mova.elamur.ai/free/en/docs/papers/cl-2003-collins-head-driven-statistical-models-natural/)
- [next → The Notion of Argument in Prepositional Phrase Attachment](https://md.mova.elamur.ai/free/en/docs/papers/cl-2006-merlo-notion-argument-prepositional-phrase-attachment/)
- [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/)

## References

26 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aluisio, Sandra M., Gisele M. Pinheiro, Marcelo Finger, Maria das Graças V. Nunes, and Stella E. O. Tagnin. 2003. The Lacio-Web Project: Overview and issues in Brazilian Portuguese corpora creation. In Proceedings of Corpus Linguistics, pages 14–21, Lancaster, UK. [find](https://scholar.google.com/scholar?q=Aluisio%2C+Sandra+M%2E%2C+Gisele+M%2E+Pinheiro%2C+Marcelo+Finger%2C+Maria+das+Grac%CC%A7as+V%2E+Nunes%2C+and+Stella+E%2E+O%2E+Tagnin%2E+2003%2E+The+Lacio%2DWeb+Project%3A+Overview+and+issues+in+Brazilian+Portuguese+corpora+creation%2E+)
2. Brill, Eric. 1995. Transformation-based error-driven learning and natural language processing: A case study in part-of-speech tagging. Computational Linguistics, 21(4):543–565. [find](https://scholar.google.com/scholar?q=Brill%2C+Eric%2E+1995%2E+Transformation%2Dbased+error%2Ddriven+learning+and+natural+language+processing%3A+A+case+study+in+part%2Dof%2Dspeech+tagging%2E+Computational+Linguistics%2C+21%284%29%3A543%E2%80%93565%2E)
3. Dudenredaktion, editor. 2004. Duden. Die deutsche Rechtschreibung (23rd ed.), volume 1 of Der Duden in zwölf Bänden. Dudenverlag, Mannheim. [find](https://scholar.google.com/scholar?q=Dudenredaktion%2C+editor%2E+2004%2E+Duden%2E+Die+deutsche+Rechtschreibung+%2823rd+ed%2E%29%2C+volume+1+of+Der+Duden+in+zwo%CC%88lf+Ba%CC%88nden%2E+Dudenverlag%2C+Mannheim%2E)
4. Dunning, Ted. 1993. Accurate methods for the statistics of surprise and coincidence. Computational Linguistics, 19(1):61–74. [find](https://scholar.google.com/scholar?q=Dunning%2C+Ted%2E+1993%2E+Accurate+methods+for+the+statistics+of+surprise+and+coincidence%2E+Computational+Linguistics%2C+19%281%29%3A61%E2%80%9374%2E)
5. Evert, Stefan and Brigitte Krenn. 2001. Methods for the qualitative evaluation of lexical association measures. In Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics, pages 188–195, Toulouse, France. [find](https://scholar.google.com/scholar?q=Evert%2C+Stefan+and+Brigitte+Krenn%2E+2001%2E+Methods+for+the+qualitative+evaluation+of+lexical+association+measures%2E+In+Proceedings+of+the+39th+Annual+Meeting+of+the+Association+for+Computational+Linguisti)
6. Firth, John R. 1957. A synopsis of linguistic theory 1930-55. In Studies in Linguistic Analysis (special volume of the Philological Society). The Philological Society, Oxford, pages 1–32. [find](https://scholar.google.com/scholar?q=Firth%2C+John+R%2E+1957%2E+A+synopsis+of+linguistic+theory+1930%2D55%2E+In+Studies+in+Linguistic+Analysis+%28special+volume+of+the+Philological+Society%29%2E+The+Philological+Society%2C+Oxford%2C+pages+1%E2%80%9332%2E)
7. Francis, Nelson W. and Henry Kucera. 1982. Frequency Analysis of English Usage: Lexicon and Grammar. Houghton Mifflin, New York. [find](https://scholar.google.com/scholar?q=Francis%2C+Nelson+W%2E+and+Henry+Kucera%2E+1982%2E+Frequency+Analysis+of+English+Usage%3A+Lexicon+and+Grammar%2E+Houghton+Mifflin%2C+New+York%2E)
8. Grefenstette, Gregory. 1999. Tokenization. In Hans van Halteren, editor, Syntactic Wordclass Tagging. Kluwer Academic Publishers, Dordrecht, pages 117–133. [find](https://scholar.google.com/scholar?q=Grefenstette%2C+Gregory%2E+1999%2E+Tokenization%2E+In+Hans+van+Halteren%2C+editor%2C+Syntactic+Wordclass+Tagging%2E+Kluwer+Academic+Publishers%2C+Dordrecht%2C+pages+117%E2%80%93133%2E)
9. Grefenstette, Gregory and Pasi Tapanainen. 1994. What is a word, what is a sentence? Problems of tokenization. In Proceedings of the 3rd International Conference on Computational Lexicography, pages 79–87, Budapest, Hungary. [find](https://scholar.google.com/scholar?q=Grefenstette%2C+Gregory+and+Pasi+Tapanainen%2E+1994%2E+What+is+a+word%2C+what+is+a+sentence%3F+Problems+of+tokenization%2E+In+Proceedings+of+the+3rd+International+Conference+on+Computational+Lexicography%2C+pages+7)
10. Kiss, Tibor and Jan Strunk. 2002a. Scaled log likelihood ratios for the detection of abbreviations in text corpora. In Proceedings of COLING 2002, pages 1228–1232, Taipei, Taiwan. [find](https://scholar.google.com/scholar?q=Kiss%2C+Tibor+and+Jan+Strunk%2E+2002a%2E+Scaled+log+likelihood+ratios+for+the+detection+of+abbreviations+in+text+corpora%2E+In+Proceedings+of+COLING+2002%2C+pages+1228%E2%80%931232%2C+Taipei%2C+Taiwan%2E)
11. Kiss, Tibor and Jan Strunk. 2002b. Viewing sentence boundary detection as collocation identification. In Proceedings of KONVENS 2002, pages 75–82, Saarbrücken, Germany. [find](https://scholar.google.com/scholar?q=Kiss%2C+Tibor+and+Jan+Strunk%2E+2002b%2E+Viewing+sentence+boundary+detection+as+collocation+identification%2E+In+Proceedings+of+KONVENS+2002%2C+pages+75%E2%80%9382%2C+Saarbru%CC%88cken%2C+Germany%2E)
12. Manning, Christopher D. and Hinrich Schütze. 1999. Foundations of Statistical Natural Language Processing. The MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=Manning%2C+Christopher+D%2E+and+Hinrich+Schu%CC%88tze%2E+1999%2E+Foundations+of+Statistical+Natural+Language+Processing%2E+The+MIT+Press%2C+Cambridge%2C+MA%2E)
13. Marcus, Mitchell P., Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330. [find](https://scholar.google.com/scholar?q=Marcus%2C+Mitchell+P%2E%2C+Beatrice+Santorini%2C+and+Mary+Ann+Marcinkiewicz%2E+1993%2E+Building+a+large+annotated+corpus+of+English%3A+The+Penn+Treebank%2E+Computational+Linguistics%2C+19%282%29%3A313%E2%80%93330%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/bank-1993-marcus-building-large-annotated-corpus-english/)
14. Mikheev, Andrei. 2000. Tagging sentence boundaries. In Proceedings of ANLP-NAACL 2000, pages 264–271, Seattle, Washington. [find](https://scholar.google.com/scholar?q=Mikheev%2C+Andrei%2E+2000%2E+Tagging+sentence+boundaries%2E+In+Proceedings+of+ANLP%2DNAACL+2000%2C+pages+264%E2%80%93271%2C+Seattle%2C+Washington%2E)
15. Mikheev, Andrei. 2002. Periods, capitalized words, etc. Computational Linguistics, 28(3):289–318. [find](https://scholar.google.com/scholar?q=Mikheev%2C+Andrei%2E+2002%2E+Periods%2C+capitalized+words%2C+etc%2E+Computational+Linguistics%2C+28%283%29%3A289%E2%80%93318%2E)
16. Müller, Hans, V. Amerl, and G. Natalis. 1980. Worterkennungsverfahren als Grundlage einer Universalmethode zur automatischen Segmentierung von Texten in Sätze. Ein Verfahren zur maschinellen Satzgrenzenbestimmung im Englischen. Sprache und Datenverarbeitung, 4(1):46–64. [find](https://scholar.google.com/scholar?q=Mu%CC%88ller%2C+Hans%2C+V%2E+Amerl%2C+and+G%2E+Natalis%2E+1980%2E+Worterkennungsverfahren+als+Grundlage+einer+Universalmethode+zur+automatischen+Segmentierung+von+Texten+in+Sa%CC%88tze%2E+Ein+Verfahren+zur+maschinellen+Satzgre)
17. Nunberg, Geoffrey. 1990. The Linguistics of Punctuation, volume 18 of CSLI Lecture Notes. CSLI Publications, Stanford, CA. [find](https://scholar.google.com/scholar?q=Nunberg%2C+Geoffrey%2E+1990%2E+The+Linguistics+of+Punctuation%2C+volume+18+of+CSLI+Lecture+Notes%2E+CSLI+Publications%2C+Stanford%2C+CA%2E)
18. Palmer, David D. and Marti A. Hearst. 1997. Adaptive multilingual sentence boundary disambiguation. Computational Linguistics, 23(2):241–267. [find](https://scholar.google.com/scholar?q=Palmer%2C+David+D%2E+and+Marti+A%2E+Hearst%2E+1997%2E+Adaptive+multilingual+sentence+boundary+disambiguation%2E+Computational+Linguistics%2C+23%282%29%3A241%E2%80%93267%2E)
19. Reynar, Jeffrey C. and Adwait Ratnaparkhi. 1997. A maximum entropy approach to identifying sentence boundaries. In Proceedings of the Fifth ACL Conference on Applied Natural Language Processing, pages 16–19, Washington, DC. [find](https://scholar.google.com/scholar?q=Reynar%2C+Jeffrey+C%2E+and+Adwait+Ratnaparkhi%2E+1997%2E+A+maximum+entropy+approach+to+identifying+sentence+boundaries%2E+In+Proceedings+of+the+Fifth+ACL+Conference+on+Applied+Natural+Language+Processing%2C+pages)
20. Riley, Michael D. 1989. Some applications of tree-based modeling to speech and language indexing. In Proceedings of the DARPA Speech and Natural Language Workshop, pages 339–352, Cape Cod, MA. [find](https://scholar.google.com/scholar?q=Riley%2C+Michael+D%2E+1989%2E+Some+applications+of+tree%2Dbased+modeling+to+speech+and+language+indexing%2E+In+Proceedings+of+the+DARPA+Speech+and+Natural+Language+Workshop%2C+pages+339%E2%80%93352%2C+Cape+Cod%2C+MA%2E)
21. Silla Jr., Carlos N. and Celso A. A. Kaestner. 2004. An analysis of sentence boundary detection systems for English and Portuguese documents. In Proceedings of CICLing 2004, pages 135–141, Seoul, Korea. [find](https://scholar.google.com/scholar?q=Silla+Jr%2E%2C+Carlos+N%2E+and+Celso+A%2E+A%2E+Kaestner%2E+2004%2E+An+analysis+of+sentence+boundary+detection+systems+for+English+and+Portuguese+documents%2E+In+Proceedings+of+CICLing+2004%2C+pages+135%E2%80%93141%2C+Seoul%2C+Kore)
22. Silla Jr., Carlos N., Jaime Dalla Valle Jr., and Celso A. A. Kaestner. 2003. Detecção automática de sentenças com o uso de expressões regulares. In Proceedings of CBComp 2003, pages 548–560, Itajaı́, Brazil. [find](https://scholar.google.com/scholar?q=Silla+Jr%2E%2C+Carlos+N%2E%2C+Jaime+Dalla+Valle+Jr%2E%2C+and+Celso+A%2E+A%2E+Kaestner%2E+2003%2E+Detecc%CC%A7a%CC%83o+automa%CC%81tica+de+sentenc%CC%A7as+com+o+uso+de+expresso%CC%83es+regulares%2E+In+Proceedings+of+CBComp+2003%2C+pages+548%E2%80%93560%2C+Itaj)
23. Stamatatos, Efstathios, Nikos Fakotakis, and George K. Kokkinakis. 1999. Automatic extraction of rules for sentence boundary disambiguation. In Proceedings of the Workshop on Machine Learning in Human Language Technology, pages 88–92, Chania, Greece. [find](https://scholar.google.com/scholar?q=Stamatatos%2C+Efstathios%2C+Nikos+Fakotakis%2C+and+George+K%2E+Kokkinakis%2E+1999%2E+Automatic+extraction+of+rules+for+sentence+boundary+disambiguation%2E+In+Proceedings+of+the+Workshop+on+Machine+Learning+in+Human)
24. Strunk, Jan, Carlos N. Silla Jr., and Celso A. A. Kaestner. 2006. A comparative evaluation of a new unsupervised sentence boundary detection approach on documents in English and Portuguese. In Proceedings of CICLing 2006, pages 132–143, Mexico City, Mexico. van Rijsbergen, Cornelis J. 1979. Informat… [find](https://scholar.google.com/scholar?q=Strunk%2C+Jan%2C+Carlos+N%2E+Silla+Jr%2E%2C+and+Celso+A%2E+A%2E+Kaestner%2E+2006%2E+A+comparative+evaluation+of+a+new+unsupervised+sentence+boundary+detection+approach+on+documents+in+English+and+Portuguese%2E+In+Proceed)
25. Walker, Daniel J., David E. Clements, Maki Darwin, and Jan W. Amtrup. 2001. Sentence boundary detection: A comparison of paradigms for improving MT quality. In Proceedings of the MT Summit VIII, Santiago de Compostela, Spain. [find](https://scholar.google.com/scholar?q=Walker%2C+Daniel+J%2E%2C+David+E%2E+Clements%2C+Maki+Darwin%2C+and+Jan+W%2E+Amtrup%2E+2001%2E+Sentence+boundary+detection%3A+A+comparison+of+paradigms+for+improving+MT+quality%2E+In+Proceedings+of+the+MT+Summit+VIII%2C+Santi)
26. Willmann, Helmut and Heinz Messinger, editors. 1996. Langenscheidts Großwörterbuch Englisch. Der Kleine Muret-Sanders (7th ed.), volume I: English– German. Langenscheidt, Berlin/Munich. [find](https://scholar.google.com/scholar?q=Willmann%2C+Helmut+and+Heinz+Messinger%2C+editors%2E+1996%2E+Langenscheidts+Gro%C3%9Fwo%CC%88rterbuch+Englisch%2E+Der+Kleine+Muret%2DSanders+%287th+ed%2E%29%2C+volume+I%3A+English%E2%80%93+German%2E+Langenscheidt%2C+Berlin%2FMunich%2E)