[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/pavlick-2015-ppdb-2-0-better-paraphrase/) [home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/mtrans/) / PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification # PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification **Authors:** [Ellie Pavlick](https://md.mova.elamur.ai/free/en/docs/papers/authors/ellie-pavlick/), [Pushpendre Rastogi](https://md.mova.elamur.ai/free/en/docs/papers/authors/pushpendre-rastogi/), [Juri Ganitkevitch](https://md.mova.elamur.ai/free/en/docs/papers/authors/juri-ganitkevitch/), [Benjamin Van Durme](https://md.mova.elamur.ai/free/en/docs/papers/authors/benjamin-van-durme/), [Chris Callison-Burch](https://md.mova.elamur.ai/free/en/docs/papers/authors/chris-callison-burch/) · **Year:** 2015 · **Venue:** ACL-IJCNLP 2015 (short papers) **Link:** [aclanthology.org/P15-2070](https://aclanthology.org/P15-2070/) · **Licence:** [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/) — summary only [**256** citations](https://openalex.org/works?filter=cites:W2251882135) openalex · 2026-10-18[**158** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/pavlick-2015-ppdb-2-0-better-paraphrase/) of the 140673 papers whose bibliographies we read[Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)](https://aclanthology.org/volumes/P15-2/)[licence](https://creativecommons.org/licenses/by-nc-sa/3.0/) - [Publisher's page](https://aclanthology.org/P15-2070/) - [PDF](https://aclanthology.org/P15-2070.pdf) - [DOI 10.3115/v1/p15-2070](https://doi.org/10.3115/v1/p15-2070) - [OpenAlex](https://openalex.org/W2251882135) **Authors (5)** 1. [Ellie Pavlick](https://openalex.org/A5053850863) [ORCID](https://orcid.org/0000-0002-7155-5420) · h 33 · 6871 citations · 194 works 2. [Pushpendre Rastogi](https://openalex.org/A5015147112) [ORCID](https://orcid.org/0000-0003-1573-3066) · h 11 · 990 citations · 33 works 3. [Juri Ganitkevitch](https://openalex.org/A5029323348) h 14 · 1708 citations · 26 works 4. [Benjamin Van Durme](https://openalex.org/A5075825791) [ORCID](https://orcid.org/0000-0003-4328-4288) · h 51 · 9972 citations · 392 works 5. Chris Callison-Burch Citations and author metrics from openalex (2026-10-18); authors with the same name are not always told apart. ## What it is about PPDB 1.0 ranked its pivot-extracted paraphrases with a fixed heuristic, and this paper replaces the ranking with a discriminative model. The model is trained on 26,000 human judgments of phrase pairs, sampled evenly across the range of the pivot probability. Its features include the pivot probability, a rarity penalty for how often a pair was seen, and similarity from word embeddings. Its rank correlates better with the human judgments than the pivot probability (Spearman 0.41), the 1.0 heuristic (0.41) or the cosine similarity of word2vec vectors (0.46), with the supervised ranking highest of the four in the paper's figure. The release also adds fine-grained entailment labels and style annotations to each pair, and it has over 100 million paraphrases in total. The lesson is that a cheap pivot probability is a weak ranking, and that a small number of human ratings can fix much of that. ## Ideas - I1. A small count-based ranker for the translator: take a few hundred merges that the residual has already judged (kept or undone) and fit a logistic model on features the translator already has, such as alignment counts, the number of verses where the two words meet, and rarity. Store the weights in a `BTreeMap` and check that two runs agree. - I2. Keep a fixed list of human-judged merges as a held-out check. A new lexicon rule must raise the correlation on that list, measured twice, before it is accepted. The check should be kept apart from the rules that were fitted to it. - I3. Use an entailment direction as a feature of a merge. The paper's example of a noise merge is *be* with *have*; a one-way label would keep a merge only where meaning goes one way, which addresses the second open problem directly. ## For Mova The problems are open problems 2 and 5: which merges are noise, and number and person features that a merge should not lose. The ranker is a count-based stand-in for the paper's model; the word embeddings are not for Mova, only the idea that similarity can be counted from context. Summary only, as the licence is non-commercial. Topics: [Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/topics/machine-translation/) ## Cite The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.3115/v1/p15-2070](https://doi.org/10.3115/v1/p15-2070) ``` @inproceedings{pavlick2015ppdb, title = {{PPDB 2.0: Better paraphrase ranking, fine-grained entailment relations, word embeddings, and style classification}}, author = {Pavlick, Ellie and Rastogi, Pushpendre and Ganitkevitch, Juri and Durme, Benjamin Van and Callison-Burch, Chris}, booktitle = {Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)}, year = {2015}, doi = {10.3115/v1/p15-2070}, url = {https://doi.org/10.3115/v1/p15-2070} } ``` - [← previous Low-Resource Authorship Style Transfer: Can Non-Famous Authors Be Imitated?](https://md.mova.elamur.ai/free/en/docs/papers/patel-2022-low-resource-authorship-style-transfer/) - [next → An Empirical Analysis of Formality in Online Communication](https://md.mova.elamur.ai/free/en/docs/papers/pavlick-2016-empirical-analysis-formality-online-communication/) - [↑ back to Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/mtrans/) ## References 28 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Colin Bannard and Chris Callison-Burch. 2005. Paraphrasing with bilingual parallel corpora. In ACL, pages 597–604. [find](https://scholar.google.com/scholar?q=Colin+Bannard+and+Chris+Callison%2DBurch%2E+2005%2E+Paraphrasing+with+bilingual+parallel+corpora%2E+In+ACL%2C+pages+597%E2%80%93604%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/bannard-2005-paraphrasing-bilingual-parallel-corpora/) 2. Islam Beltagy, Stephen Roller, Gemma Boleda, Katrin Erk, and Raymond J Mooney. 2014. Utexas: Natural language semantics using distributional semantics and probabilistic logic. In SemEval. [find](https://scholar.google.com/scholar?q=Islam+Beltagy%2C+Stephen+Roller%2C+Gemma+Boleda%2C+Katrin+Erk%2C+and+Raymond+J+Mooney%2E+2014%2E+Utexas%3A+Natural+language+semantics+using+distributional+semantics+and+probabilistic+logic%2E+In+SemEval%2E) 3. Jonathan Berant, Ido Dagan, and Jacob Goldberger. 2011. Global learning of typed entailment rules. In ACL. [find](https://scholar.google.com/scholar?q=Jonathan+Berant%2C+Ido+Dagan%2C+and+Jacob+Goldberger%2E+2011%2E+Global+learning+of+typed+entailment+rules%2E+In+ACL%2E) 4. Rahul Bhagat, Patrick Pantel, Eduard H Hovy, and Marina Rey. 2007. LEDIR: An unsupervised algorithm for learning directionality of inference rules. In EMNLP-CoNLL, pages 161–170. [find](https://scholar.google.com/scholar?q=Rahul+Bhagat%2C+Patrick+Pantel%2C+Eduard+H+Hovy%2C+and+Marina+Rey%2E+2007%2E+LEDIR%3A+An+unsupervised+algorithm+for+learning+directionality+of+inference+rules%2E+In+EMNLP%2DCoNLL%2C+pages+161%E2%80%93170%2E) 5. Johannes Bjerva, Johan Bos, Rob van der Goot, and Malvina Nissim. 2014. The meaning factory: Formal semantics for recognizing textual entailment and determining semantic similarity. In SemEval. [find](https://scholar.google.com/scholar?q=Johannes+Bjerva%2C+Johan+Bos%2C+Rob+van+der+Goot%2C+and+Malvina+Nissim%2E+2014%2E+The+meaning+factory%3A+Formal+semantics+for+recognizing+textual+entailment+and+determining+semantic+similarity%2E+In+SemEval%2E) 6. Chris Callison-Burch. 2008. Syntactic constraints on paraphrases extracted from parallel corpora. In EMNLP, pages 196–205. [find](https://scholar.google.com/scholar?q=Chris+Callison%2DBurch%2E+2008%2E+Syntactic+constraints+on+paraphrases+extracted+from+parallel+corpora%2E+In+EMNLP%2C+pages+196%E2%80%93205%2E) 7. Tsz Ping Chan, Chris Callison-Burch, and Benjamin Van Durme. 2011. Reranking bilingually extracted paraphrases using monolingual distributional similarity. In GEMS, pages 33–42. [find](https://scholar.google.com/scholar?q=Tsz+Ping+Chan%2C+Chris+Callison%2DBurch%2C+and+Benjamin+Van+Durme%2E+2011%2E+Reranking+bilingually+extracted+paraphrases+using+monolingual+distributional+similarity%2E+In+GEMS%2C+pages+33%E2%80%9342%2E) 8. Paramveer S. Dhillon, Dean Foster, and Lyle Ungar. 2011. Multi-view learning of word embeddings via CCA. In NIPS. [find](https://scholar.google.com/scholar?q=Paramveer+S%2E+Dhillon%2C+Dean+Foster%2C+and+Lyle+Ungar%2E+2011%2E+Multi%2Dview+learning+of+word+embeddings+via+CCA%2E+In+NIPS%2E) 9. Manaal Faruqui, Jesse Dodge, Sujay K Jauhar, Chris Dyer, Eduard Hovy, and Noah A Smith. 2015. Retrofitting word vectors to semantic lexicons. In NAACL. [find](https://scholar.google.com/scholar?q=Manaal+Faruqui%2C+Jesse+Dodge%2C+Sujay+K+Jauhar%2C+Chris+Dyer%2C+Eduard+Hovy%2C+and+Noah+A+Smith%2E+2015%2E+Retrofitting+word+vectors+to+semantic+lexicons%2E+In+NAACL%2E) 10. Juri Ganitkevitch, Chris Callison-Burch, Courtney Napoles, and Benjamin Van Durme. 2011. Learning sentential paraphrases from bilingual parallel corpora for text-to-text generation. In EMNLP, pages 1168–1179. [find](https://scholar.google.com/scholar?q=Juri+Ganitkevitch%2C+Chris+Callison%2DBurch%2C+Courtney+Napoles%2C+and+Benjamin+Van+Durme%2E+2011%2E+Learning+sentential+paraphrases+from+bilingual+parallel+corpora+for+text%2Dto%2Dtext+generation%2E+In+EMNLP%2C+pages+11) 11. Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. 2013. PPDB: The paraphrase database. In NAACL-HLT, pages 758–764, Atlanta, Georgia, June. [find](https://scholar.google.com/scholar?q=Juri+Ganitkevitch%2C+Benjamin+Van+Durme%2C+and+Chris+Callison%2DBurch%2E+2013%2E+PPDB%3A+The+paraphrase+database%2E+In+NAACL%2DHLT%2C+pages+758%E2%80%93764%2C+Atlanta%2C+Georgia%2C+June%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ganitkevitch-2013-ppdb-paraphrase-database/) 12. Lushan Han, Abhay Kashyap, Tim Finin, James Mayfield, and Jonathan Weese. 2013. Umbc ebiquity-core: Semantic textual similarity systems. In Proceedings of the Second Joint Conference on Lexical and Computational Semantics, volume 1, pages 44– 52. [find](https://scholar.google.com/scholar?q=Lushan+Han%2C+Abhay+Kashyap%2C+Tim+Finin%2C+James+Mayfield%2C+and+Jonathan+Weese%2E+2013%2E+Umbc+ebiquity%2Dcore%3A+Semantic+textual+similarity+systems%2E+In+Proceedings+of+the+Second+Joint+Conference+on+Lexical+and+Co) 13. Yangfeng Ji and Jacob Eisenstein. 2013. Discriminative improvements to distributional sentence similarity. In EMNLP, pages 891–896. [find](https://scholar.google.com/scholar?q=Yangfeng+Ji+and+Jacob+Eisenstein%2E+2013%2E+Discriminative+improvements+to+distributional+sentence+similarity%2E+In+EMNLP%2C+pages+891%E2%80%93896%2E) 14. Bill MacCartney. 2009. Natural language inference. Ph.D. thesis, Citeseer. [find](https://scholar.google.com/scholar?q=Bill+MacCartney%2E+2009%2E+Natural+language+inference%2E+Ph%2ED%2E+thesis%2C+Citeseer%2E) 15. Prodromos Malakasiotis and Ion Androutsopoulos. 2011. A generate and rank approach to sentence paraphrasing. In EMNLP, pages 96–106. [find](https://scholar.google.com/scholar?q=Prodromos+Malakasiotis+and+Ion+Androutsopoulos%2E+2011%2E+A+generate+and+rank+approach+to+sentence+paraphrasing%2E+In+EMNLP%2C+pages+96%E2%80%93106%2E) 16. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In Workshop at ICLR. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Kai+Chen%2C+Greg+Corrado%2C+and+Jeffrey+Dean%2E+2013%2E+Efficient+estimation+of+word+representations+in+vector+space%2E+In+Workshop+at+ICLR%2E) 17. Ellie Pavlick and Ani Nenkova. 2015. Inducing lexical style properties for paraphrase and genre differentiation. In NAACL. [find](https://scholar.google.com/scholar?q=Ellie+Pavlick+and+Ani+Nenkova%2E+2015%2E+Inducing+lexical+style+properties+for+paraphrase+and+genre+differentiation%2E+In+NAACL%2E) 18. Ellie Pavlick, Johan Bos, Malvina Nissim, Charley Beller, Benjamin Van Durme, and Chris Callison-Burch. 2015. Adding semantics to data-driven paraphrasing. In ACL. [find](https://scholar.google.com/scholar?q=Ellie+Pavlick%2C+Johan+Bos%2C+Malvina+Nissim%2C+Charley+Beller%2C+Benjamin+Van+Durme%2C+and+Chris+Callison%2DBurch%2E+2015%2E+Adding+semantics+to+data%2Ddriven+paraphrasing%2E+In+ACL%2E) 19. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. Proceedings of the Empiricial Methods in Natural Language Processing (EMNLP 2014), 12. [find](https://scholar.google.com/scholar?q=Jeffrey+Pennington%2C+Richard+Socher%2C+and+Christopher+D+Manning%2E+2014%2E+Glove%3A+Global+vectors+for+word+representation%2E+Proceedings+of+the+Empiricial+Methods+in+Natural+Language+Processing+%28EMNLP+2014%29%2C+1) 20. Pushpendre Rastogi, Benjamin Van Durme, and Raman Arora. 2015. Multiview LSA: Representation learning via generalized CCA. In NAACL. [find](https://scholar.google.com/scholar?q=Pushpendre+Rastogi%2C+Benjamin+Van+Durme%2C+and+Raman+Arora%2E+2015%2E+Multiview+LSA%3A+Representation+learning+via+generalized+CCA%2E+In+NAACL%2E) 21. Rion Snow, Daniel Jurafsky, and Andrew Y Ng. 2004. Learning syntactic patterns for automatic hypernym discovery. In NIPS. Md. [find](https://scholar.google.com/scholar?q=Rion+Snow%2C+Daniel+Jurafsky%2C+and+Andrew+Y+Ng%2E+2004%2E+Learning+syntactic+patterns+for+automatic+hypernym+discovery%2E+In+NIPS%2E+Md%2E) 22. Arafat Sultan, Steven Bethard, and Tamara Sumner. 2014a. Back to basics for monolingual alignment: Exploiting word similarity and contextual evidence. Transactions of the Association for Computational Linguistics, 2:219–230. [find](https://scholar.google.com/scholar?q=Arafat+Sultan%2C+Steven+Bethard%2C+and+Tamara+Sumner%2E+2014a%2E+Back+to+basics+for+monolingual+alignment%3A+Exploiting+word+similarity+and+contextual+evidence%2E+Transactions+of+the+Association+for+Computational) 23. Md Arafat Sultan, Steven Bethard, and Tamara Sumner. 2014b. Dls@cu: Sentence similarity from word alignment. In SemEval. [find](https://scholar.google.com/scholar?q=Md+Arafat+Sultan%2C+Steven+Bethard%2C+and+Tamara+Sumner%2E+2014b%2E+Dls%40cu%3A+Sentence+similarity+from+word+alignment%2E+In+SemEval%2E) 24. Wei Xu, Alan Ritter, Bill Dolan, Ralph Grishman, and Colin Cherry. 2012. Paraphrasing for style. In COLING. [find](https://scholar.google.com/scholar?q=Wei+Xu%2C+Alan+Ritter%2C+Bill+Dolan%2C+Ralph+Grishman%2C+and+Colin+Cherry%2E+2012%2E+Paraphrasing+for+style%2E+In+COLING%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/xu-2012-paraphrasing-style/) 25. Wei Xu, Chris Callison-Burch, and Courtney Napoles. 2015. Problems in current text simplification research: New data can help. TACL. [find](https://scholar.google.com/scholar?q=Wei+Xu%2C+Chris+Callison%2DBurch%2C+and+Courtney+Napoles%2E+2015%2E+Problems+in+current+text+simplification+research%3A+New+data+can+help%2E+TACL%2E) 26. Xuchen Yao, Benjamin Van Durme, Chris Callison-Burch, and Peter Clark. 2013. Semi-markov phrase-based monolingual alignment. In EMNLP, pages 590–600. [find](https://scholar.google.com/scholar?q=Xuchen+Yao%2C+Benjamin+Van+Durme%2C+Chris+Callison%2DBurch%2C+and+Peter+Clark%2E+2013%2E+Semi%2Dmarkov+phrase%2Dbased+monolingual+alignment%2E+In+EMNLP%2C+pages+590%E2%80%93600%2E) 27. Mo Yu and Mark Dredze. 2014. Improving lexical embeddings with semantic knowledge. In ACL, volume 2, pages 545–550. [find](https://scholar.google.com/scholar?q=Mo+Yu+and+Mark+Dredze%2E+2014%2E+Improving+lexical+embeddings+with+semantic+knowledge%2E+In+ACL%2C+volume+2%2C+pages+545%E2%80%93550%2E) 28. Shiqi Zhao, Cheng Niu, Ming Zhou, Ting Liu, and Sheng Li. 2008. Combining multiple resources to improve SMT-based paraphrasing model. In ACL. [find](https://scholar.google.com/scholar?q=Shiqi+Zhao%2C+Cheng+Niu%2C+Ming+Zhou%2C+Ting+Liu%2C+and+Sheng+Li%2E+2008%2E+Combining+multiple+resources+to+improve+SMT%2Dbased+paraphrasing+model%2E+In+ACL%2E)