[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/aclbulk-1994-ratnaparkhi-maximum-entropy-model-prepositional-phrase/) [home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / A Maximum Entropy Model for Prepositional Phrase Attachment # A Maximum Entropy Model for Prepositional Phrase Attachment **Authors:** [Adwait Ratnaparkhi](https://md.mova.elamur.ai/free/en/docs/papers/authors/adwait-ratnaparkhi/), Jeff Reynar and [Salim Roukos](https://md.mova.elamur.ai/free/en/docs/papers/authors/salim-roukos/) · **Year:** 1994 · **Venue:** Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8–11, 1994 **Link:** [aclanthology.org/H94-1048](https://aclanthology.org/H94-1048/) · **Licence:** [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/) — summary only [**202** citations](https://openalex.org/works?filter=cites:W2015042937) openalex-snapshot · 2026-10-14[**74** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/aclbulk-1994-ratnaparkhi-maximum-entropy-model-prepositional-phrase/) of the 140673 papers whose bibliographies we read[Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994](https://aclanthology.org/volumes/H94-1/)[licence](https://creativecommons.org/licenses/by-nc-sa/3.0/) - [Publisher's page](https://aclanthology.org/H94-1048/) - [PDF](https://aclanthology.org/H94-1048.pdf) - [DOI 10.3115/1075812.1075868](https://doi.org/10.3115/1075812.1075868) - [OpenAlex](https://openalex.org/W2015042937) **Authors (3)** 1. [Adwait Ratnaparkhi](https://openalex.org/A5057069732) [ORCID](https://orcid.org/0009-0008-5535-016X) · h 20 · 3704 citations · 58 works 2. [Jeff Reynar](https://openalex.org/A5053270411) h 4 · 879 citations · 7 works 3. [Salim Roukos](https://openalex.org/A5043396055) [ORCID](https://orcid.org/0000-0003-2140-4349) · h 37 · 26073 citations · 206 works Citations and author metrics from openalex-snapshot (2026-10-14); authors with the same name are not always told apart. ## What it is about The paper models the noun-or-verb attachment of a prepositional phrase with a maximum-entropy classifier over four head words: the verb, the first noun, the preposition and the second noun. Features are questions on single words, on word classes, and on both; the word classes come from a binary tree of mutual-information clusters. Features are chosen greedily by a ranking of log-likelihood gain, not by exhaustive search. On the Wall Street Journal test events the model scores 81.6% with words and classes (77.7% with words only, 79.1% with classes only); on computer-manual text it scores 84.1%. Humans given the same four head words score 88.2%, and humans who read the whole sentence score 93.2%, so the authors locate the gap in the sentence context. ## Ideas - I1. Cluster features in `en`: build mutual-information word classes (deterministic, with `BTreeMap` ordering) from the EWT vocabulary and add class-bit features to the `en` perceptron in `en/src/parse.rs` and `en/src/ptag.rs`; measure EWT test LAS with and without them. - I2. Feature selection as an explanation: use the paper's greedy log-likelihood ranking to choose feature templates for the `en` tagger and report the chosen templates with their gain, so that every template is justified in `en explain` terms. - I3. Sentence context for attachment: the paper's gap between head words and whole sentence (88.2 to 93.2) motivates testing `en/src/ctx.rs` document and sentence features on PP attachments in EWT test. ## For Mova The code is `en/src/parse.rs`, `en/src/ptag.rs` and `en/src/ctx.rs`. The chapters are [22. Phrases: noun, adjective, adverb and prepositional phrases](https://md.mova.elamur.ai/free/en/docs/languages/en/22-phrases/) and `23-complementation.md`. Measure first: the PP-attachment accuracy of the current `en` parser on EWT test against the paper's 81.6 and its human 93.2 with the sentence, so that the room is clear before any change. ## Cite The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.3115/1075812.1075868](https://doi.org/10.3115/1075812.1075868) ``` @inproceedings{ratnaparkhi1994maximum, title = {{A Maximum Entropy Model for Prepositional Phrase Attachment}}, author = {Ratnaparkhi, Adwait and Reynar, Jeff and Roukos, Salim}, booktitle = {Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994}, year = {1994}, doi = {10.3115/1075812.1075868}, url = {https://doi.org/10.3115/1075812.1075868} } ``` - [← previous Adaptive Sentence Boundary Disambiguation](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-1994-palmer-adaptive-sentence-boundary-disambiguation/) - [next → Prepositional Phrase Attachment through a Backed-off Model](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-1995-collins-prepositional-phrase-attachment-through-backed/) - [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/) ## References 10 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Jaynes, E. T., "Information Theory and Statistical Mechanics." Phys. Rev. 106, pp. 620-630, 1957. [↗](https://doi.org/10.1103/physrev.106.620) 2. Kullback, S., Information Theory in Statistics. Wiley, New York, 1959. [find](https://scholar.google.com/scholar?q=Kullback%2C+S%2E%2C+Information+Theory+in+Statistics%2E+Wiley%2C+New+York%2C+1959%2E) 3. Darroch, J.N. and Ratcliff, D., "Generalized Iterative Scaling for Log-Linear Models", The Annals of Mathematical Statistics, Vol. 43, pp 1470-1480, 1972. [↗](https://doi.org/10.1214/aoms/1177692379) 4. Delia Pietra,S., Della Pietra, V., Mercer, R. L., Roukos, S., "Adaptive Language Modeling Using Minimum Discriminant Estimation," Proceedings oflCASSP-92, pp. 1-633-636, San Francisco, March 1992. [find](https://scholar.google.com/scholar?q=Delia+Pietra%2CS%2E%2C+Della+Pietra%2C+V%2E%2C+Mercer%2C+R%2E+L%2E%2C+Roukos%2C+S%2E%2C+%22Adaptive+Language+Modeling+Using+Minimum+Discriminant+Estimation%2C%22+Proceedings+oflCASSP%2D92%2C+pp%2E+1%2D633%2D636%2C+San+Francisco%2C+March+1992%2E) 5. Black, E., Garside, R., and Leech, G., 1993. Statistically-driven Computer Grammars of English: The IBM/Lancaster Approach. Rodopi. Atlanta, Georgia. [find](https://scholar.google.com/scholar?q=Black%2C+E%2E%2C+Garside%2C+R%2E%2C+and+Leech%2C+G%2E%2C+1993%2E+Statistically%2Ddriven+Computer+Grammars+of+English%3A+The+IBM%2FLancaster+Approach%2E+Rodopi%2E+Atlanta%2C+Georgia%2E) 6. Black, E., Jelinek, F., Lafferty, J., Magerman, D. M., Mercer, R., and Roukos, S., 1993. Towards History-based Grammars: Using Richer Models for Probabilistic Parsing. In Proceedings of the Association for Computational Linguistics, 1993. Columbus, Ohio. [↗](https://doi.org/10.48550/arxiv.cmp-lg/9405007) 7. Breiman, L., Friedman, J. H., Olshen, R. A., and Stone, C. J., 1984. Classification and Regression Trees. Wadsworth and Brooks. Pacific Grove, California. [find](https://scholar.google.com/scholar?q=Breiman%2C+L%2E%2C+Friedman%2C+J%2E+H%2E%2C+Olshen%2C+R%2E+A%2E%2C+and+Stone%2C+C%2E+J%2E%2C+1984%2E+Classification+and+Regression+Trees%2E+Wadsworth+and+Brooks%2E+Pacific+Grove%2C+California%2E) 8. Brown, P. F., Della Pietra, V. J., deSouza, P. V., Lai, J. C., and Mercer, R. L. Class-based n-gram Models of Natural Language. In Proceedings of the IBM Natural Language 1TL, March, 1990. Paris, France. [find](https://scholar.google.com/scholar?q=Brown%2C+P%2E+F%2E%2C+Della+Pietra%2C+V%2E+J%2E%2C+deSouza%2C+P%2E+V%2E%2C+Lai%2C+J%2E+C%2E%2C+and+Mercer%2C+R%2E+L%2E+Class%2Dbased+n%2Dgram+Models+of+Natural+Language%2E+In+Proceedings+of+the+IBM+Natural+Language+1TL%2C+March%2C+1990%2E+Paris%2C+Fran) 9. Hindle, D. and Rooth, M. 1990. Structural Ambiguity and Lexical Relations. In Proceedings of the June 1990 DARPA Speech and Natural Language Workshop. Hidden Valley, Pennsylvania. [find](https://scholar.google.com/scholar?q=Hindle%2C+D%2E+and+Rooth%2C+M%2E+1990%2E+Structural+Ambiguity+and+Lexical+Relations%2E+In+Proceedings+of+the+June+1990+DARPA+Speech+and+Natural+Language+Workshop%2E+Hidden+Valley%2C+Pennsylvania%2E) 10. Magerman, D., 1994. Natural Language Parsing as Statistical Pattern Recognition. Ph.D. dissertation, Stanford University, California. [↗](https://doi.org/10.48550/arxiv.cmp-lg/9405009)