[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/aclbulk-2012-read-sentence-boundary-detection-long-solved/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / Sentence Boundary Detection: A Long Solved Problem?

# Sentence Boundary Detection: A Long Solved Problem?

**Authors:** Jonathon Read, [Rebecca Dridan](https://md.mova.elamur.ai/free/en/docs/papers/authors/rebecca-dridan/), [Stephan Oepen](https://md.mova.elamur.ai/free/en/docs/papers/authors/stephan-oepen/), Lars Jørgen Solberg · **Year:** 2012 · **Venue:** Proceedings of COLING 2012: Posters **Link:** [aclanthology.org/C12-2096](https://aclanthology.org/C12-2096/) · **Licence:** [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/) — summary only

[**55** citations](https://openalex.org/works?filter=cites:W2250978584) openalex · 2026-10-18[**31** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/aclbulk-2012-read-sentence-boundary-detection-long-solved/) of the 140673 papers whose bibliographies we read[Proceedings of COLING 2012: Posters](https://aclanthology.org/volumes/C12-2/)[licence](https://creativecommons.org/licenses/by-nc-sa/3.0/)

- [Publisher's page](https://aclanthology.org/C12-2096/)
- [PDF](https://aclanthology.org/C12-2096.pdf)
- [OpenAlex](https://openalex.org/W2250978584)

**Authors (4)**

1. [Jonathon Read](https://openalex.org/A5037342005) h 14 · 1128 citations · 38 works
2. [Rebecca Dridan](https://openalex.org/A5043168719) h 8 · 235 citations · 25 works
3. [Stephan Oepen](https://openalex.org/A5058412010) h 36 · 3610 citations · 234 works
4. [Lars Jørgen Solberg](https://openalex.org/A5037283669) h 2 · 65 citations · 2 works

Citations and author metrics from openalex (2026-10-18); authors with the same name are not always told apart.

## What it is about

The authors review sentence boundary detection (SBD) for English and find that results are hard to compare. Earlier reports of 99% and more use narrow task definitions. They run nine publicly available SBD systems and toolkits on standard edited corpora (WSJ, Brown, GENIA) and on user-generated Web text, with precision, recall and F1 over boundary points. Out of the box, scores vary with the text type. With a generalized task, which asks for all gold boundaries in running text, performance of 99% is not generally available. Forcing sentence boundaries at paragraph breaks and normalising curly quotes to ASCII or LaTeX quotes lifts the best-case scores on the Web samples, and the authors report that this gain is large for the user-generated text.

## Ideas

- I1. `en/src/tok.rs`: force a sentence boundary at a paragraph break and normalise curly quotes before splitting, as the paper's best-case setting does. Measure F1 over boundary points on EWT test and report the change per genre.
- I2. Report precision, recall and F1 over boundary points in the Mova sentence splitter, not only accuracy, since the paper shows accuracy hides the error on rare boundaries.
- I3. Use the paper's generalized task (all gold boundaries in running text) as the metric for the `en parse-eval` input stage, and count how many parse errors begin with a missed boundary.

## For Mova

The code is `en/src/tok.rs`, and the command for the metric is `en tok-eval`. The chapter is [07. Punctuation, capital letters, abbreviations and numbers](https://md.mova.elamur.ai/free/en/docs/languages/en/07-punctuation-capitals-numbers/). The Web and GENIA samples are research-only or NC data, so use them as tests only. Measure first on EWT test with F1 over boundary points, twice, to check the numbers are stable.

## Cite

The paper itself, not this card. BibTeX from the library's record of the work.

```
@inproceedings{read2012sentence,
  title = {{Sentence Boundary Detection: A Long Solved Problem?}},
  author = {Read, Jonathon and Dridan, Rebecca and Oepen, Stephan and Solberg, Lars Jørgen},
  booktitle = {Proceedings of COLING 2012: Posters},
  year = {2012},
  url = {https://aclanthology.org/C12-2096/}
}
```

- [← previous Parser Showdown at the Wall Street Corral: An Empirical Investigation of Error Types in Parser Output](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2012-kummerfeld-parser-showdown-wall-street-corral/)
- [next → Coarse Lexical Semantic Annotation with Supersenses: An Arabic Case Study](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2012-schneider-coarse-lexical-semantic-annotation-supersenses/)
- [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/)

## References

19 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bird, S., Klein, E., and Loper, E. (2009). Natural Language Processing with Python. O’Reilly, Beijing. [find](https://scholar.google.com/scholar?q=Bird%2C+S%2E%2C+Klein%2C+E%2E%2C+and+Loper%2C+E%2E+%282009%29%2E+Natural+Language+Processing+with+Python%2E+O%E2%80%99Reilly%2C+Beijing%2E)
2. Dridan, R. and Oepen, S. (2012). Tokenization. Returning to a long solved problem. A survey, contrastive experiment, recommendations, and toolkit. In Proceedings of the 50th Meeting of the Association for Computational Linguistics, page 378 – 382, Jeju, Republic of Korea. [find](https://scholar.google.com/scholar?q=Dridan%2C+R%2E+and+Oepen%2C+S%2E+%282012%29%2E+Tokenization%2E+Returning+to+a+long+solved+problem%2E+A+survey%2C+contrastive+experiment%2C+recommendations%2C+and+toolkit%2E+In+Proceedings+of+the+50th+Meeting+of+the+Association) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2012-dridan-tokenization-returning-long-solved-problem/)
3. Francis, W. N. and Kucera, H. (1982). Frequency Analysis of English Usage. Houghton Mifflin Co., New York. [find](https://scholar.google.com/scholar?q=Francis%2C+W%2E+N%2E+and+Kucera%2C+H%2E+%281982%29%2E+Frequency+Analysis+of+English+Usage%2E+Houghton+Mifflin+Co%2E%2C+New+York%2E)
4. Gillick, D. (2009). Sentence boundary detection and the problem with the U.S. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Short Papers, page 241 – 244, Boulder, CO, USA. [find](https://scholar.google.com/scholar?q=Gillick%2C+D%2E+%282009%29%2E+Sentence+boundary+detection+and+the+problem+with+the+U%2ES%2E+In+Proceedings+of+Human+Language+Technologies%3A+The+2009+Annual+Conference+of+the+North+American+Chapter+of+the+Association) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2009-gillick-sentence-boundary-detection-problem-u/)
5. Grefenstette, G. and Tapanainen, P. (1994). What is a word, what is a sentence? Problems of tokenization. In Proceedings of the 3rd Conference on Computational Lexicography and Text Research, page 79 – 87, Budapest, Hungary. [find](https://scholar.google.com/scholar?q=Grefenstette%2C+G%2E+and+Tapanainen%2C+P%2E+%281994%29%2E+What+is+a+word%2C+what+is+a+sentence%3F+Problems+of+tokenization%2E+In+Proceedings+of+the+3rd+Conference+on+Computational+Lexicography+and+Text+Research%2C+page+79+)
6. Kim, J.-D., Ohta, T., Teteisi, Y., and Tsujii, J. (2003). GENIA corpus — a semantically annotated corpus for bio-textmining. Bioinformatics, 19:i180 – i182. [find](https://scholar.google.com/scholar?q=Kim%2C+J%2E%2DD%2E%2C+Ohta%2C+T%2E%2C+Teteisi%2C+Y%2E%2C+and+Tsujii%2C+J%2E+%282003%29%2E+GENIA+corpus+%E2%80%94+a+semantically+annotated+corpus+for+bio%2Dtextmining%2E+Bioinformatics%2C+19%3Ai180+%E2%80%93+i182%2E)
7. Kiss, T. and Strunk, J. (2002). Viewing sentence boundary detection as collocation identification. In Proceedings of 6. Konferenz zur Varabeitung natürlicher Sprache, page 75 – 82, Saarbrücken, Germany. [find](https://scholar.google.com/scholar?q=Kiss%2C+T%2E+and+Strunk%2C+J%2E+%282002%29%2E+Viewing+sentence+boundary+detection+as+collocation+identification%2E+In+Proceedings+of+6%2E+Konferenz+zur+Varabeitung+nat%C3%BCrlicher+Sprache%2C+page+75+%E2%80%93+82%2C+Saarbr%C3%BCcken%2C+German)
8. Kiss, T. and Strunk, J. (2006). Unsupervised multilingual sentence boundary detection. Computational Linguistics, 32(4):485 – 525. [find](https://scholar.google.com/scholar?q=Kiss%2C+T%2E+and+Strunk%2C+J%2E+%282006%29%2E+Unsupervised+multilingual+sentence+boundary+detection%2E+Computational+Linguistics%2C+32%284%29%3A485+%E2%80%93+525%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/cl-2006-kiss-unsupervised-multilingual-sentence-boundary-detection/)
9. Marcus, M., Santorini, B., and Marcinkiewicz, M. A. (1993). Building a large annotated corpora of English: The Penn Treebank. Computational Linguistics, 19:313 – 330. [find](https://scholar.google.com/scholar?q=Marcus%2C+M%2E%2C+Santorini%2C+B%2E%2C+and+Marcinkiewicz%2C+M%2E+A%2E+%281993%29%2E+Building+a+large+annotated+corpora+of+English%3A+The+Penn+Treebank%2E+Computational+Linguistics%2C+19%3A313+%E2%80%93+330%2E)
10. Mikheev, A. (2000). Tagging sentence boundaries. In Proceedings of the 1st Meeting of the North American Chapter of the Association for Computational Linguistics, page 264 – 271, Seattle, WA, USA. [find](https://scholar.google.com/scholar?q=Mikheev%2C+A%2E+%282000%29%2E+Tagging+sentence+boundaries%2E+In+Proceedings+of+the+1st+Meeting+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%2C+page+264+%E2%80%93+271%2C+Seattle%2C+WA%2C+USA%2E)
11. Mikheev, A. (2002). Periods, capitalized words, etc. Computational Linguistics, 28(3):289 – 318. [find](https://scholar.google.com/scholar?q=Mikheev%2C+A%2E+%282002%29%2E+Periods%2C+capitalized+words%2C+etc%2E+Computational+Linguistics%2C+28%283%29%3A289+%E2%80%93+318%2E)
12. Morante, R. and Blanco, E. (2012). *SEM 2012 shared task: Resolving the scope and focus of negation. In Proceedings of the 1st Joint Conference on Lexical and Computational Semantics, page 265 – 274, Montréal, Canada. [find](https://scholar.google.com/scholar?q=Morante%2C+R%2E+and+Blanco%2C+E%2E+%282012%29%2E+%2ASEM+2012+shared+task%3A+Resolving+the+scope+and+focus+of+negation%2E+In+Proceedings+of+the+1st+Joint+Conference+on+Lexical+and+Computational+Semantics%2C+page+265+%E2%80%93+274%2C+)
13. Nunberg, G. (1990). The Linguistics of Punctuation. Number 18 in Lecture Notes. CSLI Publications, Stanford, CA. [find](https://scholar.google.com/scholar?q=Nunberg%2C+G%2E+%281990%29%2E+The+Linguistics+of+Punctuation%2E+Number+18+in+Lecture+Notes%2E+CSLI+Publications%2C+Stanford%2C+CA%2E)
14. Palmer, D. D. and Hearst, M. A. (1997). Adaptive multilingual sentence boundary disambiguation. Computational Linguistics, 23(2):242 – 267. [find](https://scholar.google.com/scholar?q=Palmer%2C+D%2E+D%2E+and+Hearst%2C+M%2E+A%2E+%281997%29%2E+Adaptive+multilingual+sentence+boundary+disambiguation%2E+Computational+Linguistics%2C+23%282%29%3A242+%E2%80%93+267%2E)
15. Read, J., Flickinger, D., Dridan, R., Oepen, S., and Øvrelid, L. (2012). The WeSearch Corpus, Treebank, and Treecache. A comprehensive sample of user-generated content. In Proceedings of the 8th International Conference on Language Resources and Evaluation, Istanbul, Turkey. [find](https://scholar.google.com/scholar?q=Read%2C+J%2E%2C+Flickinger%2C+D%2E%2C+Dridan%2C+R%2E%2C+Oepen%2C+S%2E%2C+and+%C3%98vrelid%2C+L%2E+%282012%29%2E+The+WeSearch+Corpus%2C+Treebank%2C+and+Treecache%2E+A+comprehensive+sample+of+user%2Dgenerated+content%2E+In+Proceedings+of+the+8th+Inter)
16. Reynar, J. C. and Ratnaparkhi, A. (1997). A maximum entropy approach to identifying sentence boundaries. In Proceedings of the 5th Conference on Applied Natural Language Processing, page 16 – 19, Washington, D.C., USA. [find](https://scholar.google.com/scholar?q=Reynar%2C+J%2E+C%2E+and+Ratnaparkhi%2C+A%2E+%281997%29%2E+A+maximum+entropy+approach+to+identifying+sentence+boundaries%2E+In+Proceedings+of+the+5th+Conference+on+Applied+Natural+Language+Processing%2C+page+16+%E2%80%93+19%2C+Wash)
17. Riley, M. D. (1989). Some applications of tree-based modelling to speech and language. In Proceedings of the DARPA Speech and Natural Language Workshop, page 339 – 352. [find](https://scholar.google.com/scholar?q=Riley%2C+M%2E+D%2E+%281989%29%2E+Some+applications+of+tree%2Dbased+modelling+to+speech+and+language%2E+In+Proceedings+of+the+DARPA+Speech+and+Natural+Language+Workshop%2C+page+339+%E2%80%93+352%2E)
18. Walker, D. J., Clements, D. E., Darwin, M., and Amtrup, J. W. (2001). Sentence boundary detection: A comparison of paradigms for improving MT quality. In Proceedings of the MT Summit VIII, Santiago de Compostela, Spain. [find](https://scholar.google.com/scholar?q=Walker%2C+D%2E+J%2E%2C+Clements%2C+D%2E+E%2E%2C+Darwin%2C+M%2E%2C+and+Amtrup%2C+J%2E+W%2E+%282001%29%2E+Sentence+boundary+detection%3A+A+comparison+of+paradigms+for+improving+MT+quality%2E+In+Proceedings+of+the+MT+Summit+VIII%2C+Santiago+de)
19. Ytrestøl, G., Oepen, S., and Flickinger, D. (2009). Extracting and annotating Wikipedia subdomains. In Proceedings of the 7th International Workshop on Treebanks and Linguistic Theories, page 185 – 197, Groningen, The Netherlands. [find](https://scholar.google.com/scholar?q=Ytrest%C3%B8l%2C+G%2E%2C+Oepen%2C+S%2E%2C+and+Flickinger%2C+D%2E+%282009%29%2E+Extracting+and+annotating+Wikipedia+subdomains%2E+In+Proceedings+of+the+7th+International+Workshop+on+Treebanks+and+Linguistic+Theories%2C+page+185+%E2%80%93+19)