[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/aclbulk-2021-ratnaparkhi-resolving-prepositional-phrase-attachment-ambiguities/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / Resolving Prepositional Phrase Attachment Ambiguities with Contextualized Word Embeddings

# Resolving Prepositional Phrase Attachment Ambiguities with Contextualized Word Embeddings

**Authors:** [Adwait Ratnaparkhi](https://md.mova.elamur.ai/free/en/docs/papers/authors/adwait-ratnaparkhi/) and Atul Kumar · **Year:** 2021 · **Venue:** Proceedings of the 18th International Conference on Natural Language Processing (ICON) **Link:** [aclanthology.org/2021.icon-main.40](https://aclanthology.org/2021.icon-main.40/) · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)

[**1** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/aclbulk-2021-ratnaparkhi-resolving-prepositional-phrase-attachment-ambiguities/) of the 140673 papers whose bibliographies we read[Proceedings of the 18th International Conference on Natural Language Processing (ICON)](https://aclanthology.org/volumes/2021.icon-main/)[licence](https://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://aclanthology.org/2021.icon-main.40/)
- [PDF](https://aclanthology.org/2021.icon-main.40.pdf)

**Authors (2)**

1. Adwait Ratnaparkhi
2. Atul Kumar

Citations and author metrics from OpenAlex (); authors with the same name are not always told apart.

## What it is about

The paper treats PP attachment as a choice among candidate heads (a verb and a noun, or several heads) for a preposition, with the child noun as context. It fine-tunes a contextual embedding model (RoBERTa or BERT) on two English data sets, RRR (binary verb-or-noun) and BLBG (a list of candidate heads). The test sets have 3097 and 1951 items. On RRR the best model reaches 88.8% against 88.1% for the earlier best result, a gain of 0.7 points. On BLBG the head-words-only model is 1.7 points above the best earlier result, and adding the full sentence as context gives a further gain: the paper reports a total of 4.1 points over that result. Full sentence context adds about 2% to 3% accuracy, without any syntactic annotation. The model has at most 126M parameters.

## Ideas

- I1. `en/src/ctx.rs` and `en/src/parse.rs`: add the lemmas of the whole sentence as a bag of features for the PP attachment choice, with no syntax. Measure accuracy on EWT test PP attachments with and without the bag, on the two-fold check, and run twice to confirm the numbers are identical.
- I2. A transparent head-triple table: counts of (candidate head, preposition, child noun) with back-off to (head, preposition) and then to the preposition alone, printed by `en explain` with the counts that decided. This is the cheap analogue of the contextual model. Measure on EWT test against the perceptron.
- I3. Silver labels from the teacher: a large model proposes attachment labels for unlabelled English PP sentences in `annot` offline, and a Rust gate keeps only the ones that agree with the parser's tree evidence. The count table is trained on that silver set. The teacher never answers at test time. Measure the table's accuracy on gold EWT test.

## For Mova

The code is `en/src/parse.rs` and `en/src/ctx.rs`. The chapter is [22. Phrases: noun, adjective, adverb and prepositional phrases](https://md.mova.elamur.ai/free/en/docs/languages/en/22-phrases/). The neural model itself is not a Mova component; only the context idea and the evaluation habit carry over. The RRR and BLBG sets are derived from the Penn Treebank, so they are test only and must not enter any training file. Any figure for Mova must come from EWT test.

## Cite

The paper itself, not this card. BibTeX from the library's record of the work.

```
@inproceedings{ratnaparkhi2021resolving,
  title = {{Resolving Prepositional Phrase Attachment Ambiguities with Contextualized Word Embeddings}},
  author = {Ratnaparkhi, Adwait and Kumar, Atul},
  booktitle = {Proceedings of the 18th International Conference on Natural Language Processing (ICON)},
  year = {2021},
  url = {https://aclanthology.org/2021.icon-main.40/}
}
```

- [← previous Sentence Boundary Detection on Line Breaks in Japanese](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2020-hayashibe-sentence-boundary-detection-line-breaks/)
- [next → Cross-Genre Learning for Old English Poetry POS Tagging](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2025-miani-cross-genre-learning-old-english/)
- [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/)

## References

19 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. C. Baker, C. Fillmore, and J. Lowe. 1998. The Berkeley FrameNet Project. In COLING-ACL. [find](https://scholar.google.com/scholar?q=C%2E+Baker%2C+C%2E+Fillmore%2C+and+J%2E+Lowe%2E+1998%2E+The+Berkeley+FrameNet+Project%2E+In+COLING%2DACL%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/lex-1998-baker-berkeley-framenet/)
2. Yonatan Belinkov, Tao Lei, Regina Barzilay, and Amir Globerson. 2014. Exploring compositional architectures and word vector representations for prepositional phrase attachment. Transactions of the Association for Computational Linguistics, 2:561–572. [find](https://scholar.google.com/scholar?q=Yonatan+Belinkov%2C+Tao+Lei%2C+Regina+Barzilay%2C+and+Amir+Globerson%2E+2014%2E+Exploring+compositional+architectures+and+word+vector+representations+for+prepositional+phrase+attachment%2E+Transactions+of+the+Ass) [our card](https://md.mova.elamur.ai/free/en/docs/papers/tacl-2014-belinkov-exploring-compositional-architectures-word-vector/)
3. Eric Brill and Philip Resnik. 1994. A rule-based approach to prepositional phrase attachment disambiguation. In COLING 1994. [find](https://scholar.google.com/scholar?q=Eric+Brill+and+Philip+Resnik%2E+1994%2E+A+rule%2Dbased+approach+to+prepositional+phrase+attachment+disambiguation%2E+In+COLING+1994%2E)
4. Michael Collins and James Brooks. 1995. Prepositional phrase attachment through a backed-off model. In Proceedings of the Third Workshop on Very Large Corpora. [find](https://scholar.google.com/scholar?q=Michael+Collins+and+James+Brooks%2E+1995%2E+Prepositional+phrase+attachment+through+a+backed%2Doff+model%2E+In+Proceedings+of+the+Third+Workshop+on+Very+Large+Corpora%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-1995-collins-prepositional-phrase-attachment-through-backed/)
5. Pradeep Dasigi, Waleed Ammar, Chris Dyer, and Eduard Hovy. 2017. Ontology-aware token embeddings for prepositional phrase attachment. [find](https://scholar.google.com/scholar?q=Pradeep+Dasigi%2C+Waleed+Ammar%2C+Chris+Dyer%2C+and+Eduard+Hovy%2E+2017%2E+Ontology%2Daware+token+embeddings+for+prepositional+phrase+attachment%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/onto-2017-dasigi-ontology-aware-token-embeddings-prepositional/)
6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2018%2E+BERT%3A+pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+CoRR%2C+abs%2F1810%2E04805%2E)
7. Bich-Ngoc Do and Ines Rehbein. 2020. Parsers know best: German PP attachment revisited. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2049–2061. [find](https://scholar.google.com/scholar?q=Bich%2DNgoc+Do+and+Ines+Rehbein%2E+2020%2E+Parsers+know+best%3A+German+PP+attachment+revisited%2E+In+Proceedings+of+the+28th+International+Conference+on+Computational+Linguistics%2C+pages+2049%E2%80%932061%2E)
8. Donald Hindle and Mats Rooth. 1993. Structural ambiguity and lexical relations. Computational linguistics, 19(1):103–120. [find](https://scholar.google.com/scholar?q=Donald+Hindle+and+Mats+Rooth%2E+1993%2E+Structural+ambiguity+and+lexical+relations%2E+Computational+linguistics%2C+19%281%29%3A103%E2%80%93120%2E)
9. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+RoBERTa%3A+A+robustly+optimized+BERT+pretraining+approach%2E+a)
10. Pranava Swaroop Madhyastha, Xavier Carreras, and Ariadna Quattoni. 2017. Prepositional phrase attachment over word embedding products. In Proceedings of the 15th International Conference on Parsing Technologies, pages 32–43. [find](https://scholar.google.com/scholar?q=Pranava+Swaroop+Madhyastha%2C+Xavier+Carreras%2C+and+Ariadna+Quattoni%2E+2017%2E+Prepositional+phrase+attachment+over+word+embedding+products%2E+In+Proceedings+of+the+15th+International+Conference+on+Parsing+Te) [our card](https://md.mova.elamur.ai/free/en/docs/papers/lingbulk-2017-madhyastha-prepositional-phrase-attachment-over-word/)
11. M. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Comput. Linguistics, 19:313–330. [find](https://scholar.google.com/scholar?q=M%2E+Marcus%2C+Beatrice+Santorini%2C+and+Mary+Ann+Marcinkiewicz%2E+1993%2E+Building+a+large+annotated+corpus+of+English%3A+The+Penn+Treebank%2E+Comput%2E+Linguistics%2C+19%3A313%E2%80%93330%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/bank-1993-marcus-building-large-annotated-corpus-english/)
12. G. Miller. 1995. Wordnet: a lexical database for english. Commun. ACM, 38:39–41. [find](https://scholar.google.com/scholar?q=G%2E+Miller%2E+1995%2E+Wordnet%3A+a+lexical+database+for+english%2E+Commun%2E+ACM%2C+38%3A39%E2%80%9341%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/lex-1994-miller-wordnet/)
13. Marian Olteanu and Dan Moldovan. 2005. PP-attachment disambiguation using large context. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 273–280. [find](https://scholar.google.com/scholar?q=Marian+Olteanu+and+Dan+Moldovan%2E+2005%2E+PP%2Dattachment+disambiguation+using+large+context%2E+In+Proceedings+of+Human+Language+Technology+Conference+and+Conference+on+Empirical+Methods+in+Natural+Language+)
14. Adwait Ratnaparkhi, Jeff Reynar, and Salim Roukos. 1994. A maximum entropy model for prepositional phrase attachment. In Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994. [find](https://scholar.google.com/scholar?q=Adwait+Ratnaparkhi%2C+Jeff+Reynar%2C+and+Salim+Roukos%2E+1994%2E+A+maximum+entropy+model+for+prepositional+phrase+attachment%2E+In+Human+Language+Technology%3A+Proceedings+of+a+Workshop+held+at+Plainsboro%2C+New+Je) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-1994-ratnaparkhi-maximum-entropy-model-prepositional-phrase/)
15. Jiri Stetina and Makoto Nagao. 1997. Corpus based PP attachment ambiguity resolution with a semantic dictionary. In Fifth Workshop on Very Large Corpora. [find](https://scholar.google.com/scholar?q=Jiri+Stetina+and+Makoto+Nagao%2E+1997%2E+Corpus+based+PP+attachment+ambiguity+resolution+with+a+semantic+dictionary%2E+In+Fifth+Workshop+on+Very+Large+Corpora%2E)
16. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+neural+information+processi)
17. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lho… [find](https://scholar.google.com/scholar?q=Thomas+Wolf%2C+Lysandre+Debut%2C+Victor+Sanh%2C+Julien+Chaumond%2C+Clement+Delangue%2C+Anthony+Moi%2C+Pier%2Dric+Cistac%2C+Tim+Rault%2C+R%C3%A9mi+Louf%2C+Morgan+Funtowicz%2C+Joe+Davison%2C+Sam+Shleifer%2C+Patrick+von+Platen%2C+Clara+)
18. Mo Yu, Mark Dredze, Raman Arora, and Matthew R Gormley. 2016. Embedding lexical features via low-rank tensors. In Proceedings of NAACL-HLT, pages 1019–1029. [find](https://scholar.google.com/scholar?q=Mo+Yu%2C+Mark+Dredze%2C+Raman+Arora%2C+and+Matthew+R+Gormley%2E+2016%2E+Embedding+lexical+features+via+low%2Drank+tensors%2E+In+Proceedings+of+NAACL%2DHLT%2C+pages+1019%E2%80%931029%2E)
19. Rowan Zellers, Yonatan Bisk, Roy Schwartz, and Yejin Choi. 2018. SWAG: A large-scale adversarial dataset for grounded commonsense inference. CoRR, abs/1808.05326. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Yonatan+Bisk%2C+Roy+Schwartz%2C+and+Yejin+Choi%2E+2018%2E+SWAG%3A+A+large%2Dscale+adversarial+dataset+for+grounded+commonsense+inference%2E+CoRR%2C+abs%2F1808%2E05326%2E)