[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/white-2008-more-precise-analysis-punctuation-broad/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/mtrans/) / A More Precise Analysis of Punctuation for Broad-Coverage Surface Realization with CCG

# A More Precise Analysis of Punctuation for Broad-Coverage Surface Realization with CCG

**Authors:** [Michael White](https://md.mova.elamur.ai/free/en/docs/papers/authors/michael-white/), [Rajakrishnan Rajkumar](https://md.mova.elamur.ai/free/en/docs/papers/authors/rajakrishnan-rajkumar/) · **Year:** 2008 · **Venue:** Coling 2008: Proceedings of the workshop on Grammar Engineering Across Frameworks **Link:** [aclanthology.org/W08-1703](https://aclanthology.org/W08-1703/) · **Licence:** [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/) — summary only

[**13** citations](https://openalex.org/works?filter=cites:W1964823536) openalex-snapshot · 2026-10-14[**12** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/white-2008-more-precise-analysis-punctuation-broad/) of the 140673 papers whose bibliographies we read[Coling 2008: Proceedings of the workshop on Grammar Engineering Across Frameworks](https://aclanthology.org/volumes/W08-17/)[licence](https://creativecommons.org/licenses/by-nc-sa/3.0/)

- [Publisher's page](https://aclanthology.org/W08-1703/)
- [PDF](https://aclanthology.org/W08-1703.pdf)
- [DOI 10.3115/1611546.1611549](https://doi.org/10.3115/1611546.1611549)
- [OpenAlex](https://openalex.org/W1964823536)

**Authors (2)**

1. [Michael White](https://openalex.org/A5025147385) [ORCID](https://orcid.org/0000-0002-3062-1719) · h 35 · 11560 citations · 220 works
2. [Rajakrishnan Rajkumar](https://openalex.org/A5064666736) [ORCID](https://orcid.org/0000-0003-2052-4082) · h 10 · 342 citations · 52 works

Citations and author metrics from openalex-snapshot (2026-10-14); authors with the same name are not always told apart.

## What it is about

The paper gives punctuation a lexical analysis in a broad-coverage CCG realisation grammar. In CCGbank, punctuation marks carry categories taken from their tags and sit inside binary rules, which is enough for parsing but overgenerates in realisation: a comma category can wrap a sentence in commas or repeat itself. The authors discuss a syntactic-feature solution (Briscoe 1994) and reject it for CCG, because word order there is flexible. They choose semantic features instead. Every punctuation mark, except balancing marks, gets a category whose logical form carries a discourse function such as emphasis, elaboration or apposition. Appositives are split into balanced (sentence-medial) and unbalanced (sentence-final) cases by heuristic rules, and a rule-based filter bars illegal punctuation sequences and unbalanced apposition at realisation time. The corpus gained 89 punctuation categories in the development section. The authors report about a 9-point BLEU gain over the best oracle-n-gram baseline on non-blind data, a 40% rise in exact matches, and a blind-test BLEU gain above 5.5 points with over 96% coverage on section 23.

## Ideas

- I1. A discourse-function feature on the meta token. Punctuation role (emphasis, elaboration, apposition) belongs on the meta token, and the encoder writes the mark from it. This is open problem 4 in its direct form. Derive the classes from the source side, count them per concept pair, and keep them as an enum.
- I2. Count balanced appositives in the target corpora. Measure how often a sentence-medial appositive is balanced in KJV and in WEB, and make the encoder's comma pairing a counted rule with a per-author delta.
- I3. A post-check filter with its own residual class. Reject illicit output sequences (doubled commas, a comma before a full clause) and count them as a separate class. Negative control: run the filter on shuffled tokens; it must flag most of them, or it is not a gate.
- I4. Split the punctuation residual into "mark chosen" and "mark placed". Which part dominates on KJV and WEB tells whether punctuation is a lexical or an ordering problem, and so whether it should be an author delta.

## For Mova

The code is `tools/translate/src/profile.rs` for the encoder and the punctuation residual in `tools/translate/src/measure.rs`. The ideas touch open problem 4 (boundaries as features of meta tokens) and problem 8 (style deltas). The paper's filter is a rule, so it fits the deterministic design; its statistics do not transfer and must be counted on our own data.

Topics: [Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/topics/machine-translation/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.3115/1611546.1611549](https://doi.org/10.3115/1611546.1611549)

```
@inproceedings{white2008more,
  title = {{A More Precise Analysis of Punctuation for Broad-Coverage Surface Realization with CCG}},
  author = {White, Michael and Rajkumar, Rajakrishnan},
  booktitle = {Coling 2008: Proceedings of the workshop on Grammar Engineering Across Frameworks},
  year = {2008},
  doi = {10.3115/1611546.1611549},
  url = {https://doi.org/10.3115/1611546.1611549}
}
```

- [← previous Towards broad coverage surface realization with CCG](https://md.mova.elamur.ai/free/en/docs/papers/white-2007-broad-coverage-surface-realization-ccg/)
- [next → Shake-and-Bake Translation](https://md.mova.elamur.ai/free/en/docs/papers/whitelock-1992-shake-bake-translation/)
- [↑ back to Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/mtrans/)

## References

18 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Baldridge, Jason and Geert-Jan Kruijff. 2002. Coupling CCG and Hybrid Logic Dependency Semantics. In Proc. ACL-02. [find](https://scholar.google.com/scholar?q=Baldridge%2C+Jason+and+Geert%2DJan+Kruijff%2E+2002%2E+Coupling+CCG+and+Hybrid+Logic+Dependency+Semantics%2E+In+Proc%2E+ACL%2D02%2E)
2. Baldridge, Jason. 2002. Lexically Specified Derivational Control in Combinatory Categorial Grammar. Ph.D. thesis, University of Edinburgh. [find](https://scholar.google.com/scholar?q=Baldridge%2C+Jason%2E+2002%2E+Lexically+Specified+Derivational+Control+in+Combinatory+Categorial+Grammar%2E+Ph%2ED%2E+thesis%2C+University+of+Edinburgh%2E)
3. Bayraktar, Murat, Bilge Say, and Varol Akman. 1998. An Analysis of English Punctuation: The Special Case of Comma. International Journal of Corpus Linguistics, 3(1):33–58. [↗](https://doi.org/10.1075/ijcl.3.1.03bay)
4. Boxwell, Stephen and Michael White. 2008. Projecting Propbank roles onto the CCGbank. In Proc. LREC-08. To appear. [find](https://scholar.google.com/scholar?q=Boxwell%2C+Stephen+and+Michael+White%2E+2008%2E+Projecting+Propbank+roles+onto+the+CCGbank%2E+In+Proc%2E+LREC%2D08%2E+To+appear%2E)
5. Briscoe, Ted. 1994. Parsing (with) punctuation. Technical report, Xerox, Grenoble, France. [find](https://scholar.google.com/scholar?q=Briscoe%2C+Ted%2E+1994%2E+Parsing+%28with%29+punctuation%2E+Technical+report%2C+Xerox%2C+Grenoble%2C+France%2E)
6. Doran, Christine. 1998. Incorporating Punctuation into the Sentence Grammar: A Lexicalized Tree Adjoining Grammar Perspective. Ph.D. thesis, University of Pennsylvania. [find](https://scholar.google.com/scholar?q=Doran%2C+Christine%2E+1998%2E+Incorporating+Punctuation+into+the+Sentence+Grammar%3A+A+Lexicalized+Tree+Adjoining+Grammar+Perspective%2E+Ph%2ED%2E+thesis%2C+University+of+Pennsylvania%2E)
7. Espinosa, Dominic, Michael White, and Dennis Mehay. 2008. Hypertagging: Supertagging for surface realization with CCG. In Proc. ACL-08:HLT. To appear. [find](https://scholar.google.com/scholar?q=Espinosa%2C+Dominic%2C+Michael+White%2C+and+Dennis+Mehay%2E+2008%2E+Hypertagging%3A+Supertagging+for+surface+realization+with+CCG%2E+In+Proc%2E+ACL%2D08%3AHLT%2E+To+appear%2E)
8. Forst, Martin and Ronald M. Kaplan. 2006. The importance of precise tokenizing for deep grammars. In Proc. LREC-06. [find](https://scholar.google.com/scholar?q=Forst%2C+Martin+and+Ronald+M%2E+Kaplan%2E+2006%2E+The+importance+of+precise+tokenizing+for+deep+grammars%2E+In+Proc%2E+LREC%2D06%2E)
9. Hockenmaier, Julia and Mark Steedman. 2007. CCGbank: A Corpus of CCG Derivations and Dependency Structures Extracted from the Penn Treebank. Computational Linguistics, 33(3):355–396. [↗](https://doi.org/10.1162/coli.2007.33.3.355) [our card](https://md.mova.elamur.ai/free/en/docs/papers/bank-2007-hockenmaier-ccgbank-corpus-ccg-derivations-dependency/)
10. Hogan, Deirdre, Conor Cafferkey, Aoife Cahill, and Josef van Genabith. 2007. Exploiting multi-word units in history-based probabilistic generation. In Proc. EMNLP-CoNLL-07. [find](https://scholar.google.com/scholar?q=Hogan%2C+Deirdre%2C+Conor+Cafferkey%2C+Aoife+Cahill%2C+and+Josef+van+Genabith%2E+2007%2E+Exploiting+multi%2Dword+units+in+history%2Dbased+probabilistic+generation%2E+In+Proc%2E+EMNLP%2DCoNLL%2D07%2E)
11. Langkilde-Geary, Irene. 2002. An empirical verification of coverage and correctness for a general-purpose sentence generator. In Proc. INLG-02. [find](https://scholar.google.com/scholar?q=Langkilde%2DGeary%2C+Irene%2E+2002%2E+An+empirical+verification+of+coverage+and+correctness+for+a+general%2Dpurpose+sentence+generator%2E+In+Proc%2E+INLG%2D02%2E)
12. Nakanishi, Hiroko, Yusuke Miyao, and Jun’ichi Tsujii. 2005. Probabilistic methods for disambiguation of an HPSG-based chart generator. In Proc. IWPT-05. [find](https://scholar.google.com/scholar?q=Nakanishi%2C+Hiroko%2C+Yusuke+Miyao%2C+and+Jun%E2%80%99ichi+Tsujii%2E+2005%2E+Probabilistic+methods+for+disambiguation+of+an+HPSG%2Dbased+chart+generator%2E+In+Proc%2E+IWPT%2D05%2E)
13. Nunberg, Geoffrey. 1990. The Linguistics of Punctuation. CSLI Publications, Stanford, CA. [find](https://scholar.google.com/scholar?q=Nunberg%2C+Geoffrey%2E+1990%2E+The+Linguistics+of+Punctuation%2E+CSLI+Publications%2C+Stanford%2C+CA%2E)
14. Steedman, Mark and S. Prevost. 1994. Specifying intonation from context for speech synthesis. Speech Communication, 15(1–2):139–153. [↗](https://doi.org/10.1016/0167-6393(94)90048-5)
15. Steedman, Mark. 2000. The syntactic process. MIT Press, Cambridge, MA, USA. [↗](https://doi.org/10.7551/mitpress/6591.001.0001)
16. White, Michael, Rajakrishnan Rajkumar, and Scott Martin. 2007. Towards broad coverage surface realization with CCG. In Proc. of the Workshop on Using Corpora for NLG: Language Generation and Machine Translation (UCNLG+MT). [find](https://scholar.google.com/scholar?q=White%2C+Michael%2C+Rajakrishnan+Rajkumar%2C+and+Scott+Martin%2E+2007%2E+Towards+broad+coverage+surface+realization+with+CCG%2E+In+Proc%2E+of+the+Workshop+on+Using+Corpora+for+NLG%3A+Language+Generation+and+Machine+T) [our card](https://md.mova.elamur.ai/free/en/docs/papers/white-2007-broad-coverage-surface-realization-ccg/)
17. White, Michael. 1995. Presenting punctuation. In Proceedings of the Fifth European Workshop on Natural Language Generation, pages 107–125. [find](https://scholar.google.com/scholar?q=White%2C+Michael%2E+1995%2E+Presenting+punctuation%2E+In+Proceedings+of+the+Fifth+European+Workshop+on+Natural+Language+Generation%2C+pages+107%E2%80%93125%2E)
18. White, Michael. 2006. Efficient realization of coordinate structures in combinatory categorial grammar. Research on Language and Computation, 4(1):39– 75. [↗](https://doi.org/10.1007/s11168-006-9010-2)