[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/han-2023-prewome-exploiting-presuppositions-as-working/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / PreWoMe: Exploiting Presuppositions as Working Memory for Long Form Question Answering

# PreWoMe: Exploiting Presuppositions as Working Memory for Long Form Question Answering

**Authors:** Wookje Han, Jinsol Park and Kyungjae Lee · **Year:** 2023 · **Venue:** Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing **Link:** [aclanthology.org/2023.emnlp-main.517](https://aclanthology.org/2023.emnlp-main.517/) · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) (ACL Anthology)

[**1** citations](https://openalex.org/works?filter=cites:W4389519114) openalex-snapshot · 2026-10-14[**3** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/han-2023-prewome-exploiting-presuppositions-as-working/) of the 140673 papers whose bibliographies we read[Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing](https://aclanthology.org/volumes/2023.emnlp-main/)[licence](https://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://aclanthology.org/2023.emnlp-main.517/)
- [PDF](https://aclanthology.org/2023.emnlp-main.517.pdf)
- [DOI 10.18653/v1/2023.emnlp-main.517](https://doi.org/10.18653/v1/2023.emnlp-main.517)
- [arXiv 2310.16147](https://arxiv.org/abs/2310.16147)
- [OpenAlex](https://openalex.org/W4389519114)

**Authors (3)**

1. [Wookje Han](https://openalex.org/A5075966981) h 2 · 18 citations · 6 works
2. [Jinsol Park](https://openalex.org/A5101775012) [ORCID](https://orcid.org/0000-0001-7432-2158) · h 6 · 134 citations · 13 works
3. [Kyungjae Lee](https://openalex.org/A5100604923) [ORCID](https://orcid.org/0000-0003-0586-3748) · h 9 · 250 citations · 41 works

Citations and author metrics from openalex-snapshot (2026-10-14); authors with the same name are not always told apart.

## What it is about

PreWoMe is a three-step prompting method for GPT-3.5 and GPT-4 at temperature 0, with no parameter updates. First the model lists the presuppositions of a question, and a fixed presupposition "there is a clear and single answer" is added. Second it writes feedback, a judgement on whether a presupposition is false, which also classifies the question as normal, ambiguous or false-presupposition, and an action. Third it answers with the question, the feedback and the action as guidance. On ASQA (ambiguous), the false-presupposition subset of (QA)2 and BioASQ (normal), GPT-4 with PreWoMe beats the plain prompt on all three and beats chain-of-thought on the misleading sets. Manual checks show that 391 of 395 extracted presuppositions (about 99%) were indeed assumed. The feedback classifies normal questions correctly in 74.9% of cases and false-presupposition questions in 74.4%, but ambiguous questions only in 35.9%. Wrong feedback hurts: GPT-3.5 copies a bad action and answers from it.

## Ideas

- I1. Presupposition triggers as rules: a lexicon of trigger patterns (definite descriptions such as "the only", ordinals such as "the first", factive verbs) in [33. Implicature and presupposition](https://md.mova.elamur.ai/free/en/docs/languages/en/33-implicature-and-presupposition/), emitting presupposition frames in `prag`. Check each frame against the world and measure the precision on the school questions.
- I2. The "single clear answer" presupposition becomes a count check: if the world holds two or more answers for the question slot, flag ambiguity. The paper's weak point is exactly this class, so measure the recall of the flag separately.
- I3. Make the feedback a rule-based label with a confusion matrix like the paper's Table 3, so that every misclassification can be explained by the rule that fired.
- I4. A quality gate before answering: do not pass feedback to the answerer unless it passes a check against the world. Negative control: random feedback must score worse than no feedback.

## For Mova

The chapters are [33. Implicature and presupposition](https://md.mova.elamur.ai/free/en/docs/languages/en/33-implicature-and-presupposition/) and [26. Questions](https://md.mova.elamur.ai/free/en/docs/languages/en/26-questions/). The code is in the `prag` crate and `world/src/qa.rs`. The measures are the precision of the presupposition rules on a held-out set, the recall of the ambiguity flag, and the accuracy of the answer with and without the gate. Run each twice.

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.18653/v1/2023.emnlp-main.517](https://doi.org/10.18653/v1/2023.emnlp-main.517) · [arXiv 2310.16147](https://arxiv.org/abs/2310.16147)

```
@inproceedings{han2023prewome,
  title = {{PreWoMe: Exploiting Presuppositions as Working Memory for Long Form Question Answering}},
  author = {Han, Wookje and Park, Jinsol and Lee, Kyungjae},
  booktitle = {Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing},
  year = {2023},
  doi = {10.18653/v1/2023.emnlp-main.517},
  eprint = {2310.16147},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2023.emnlp-main.517}
}
```

- [← previous Comparing PTB and UD information for PDTB discourse connective identification](https://md.mova.elamur.ai/free/en/docs/papers/han-2020-comparing-ptb-ud-information-pdtb/)
- [next → Uniform Information Density and Syntactic Reduction: Revisiting that-Mentioning in English Complement Clauses](https://md.mova.elamur.ai/free/en/docs/papers/hao-2025-uniform-information-density-syntactic-reduction-revisiting/)
- [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/)

## References

31 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Reinald Kim Amplayo, Kellie Webster, Michael Collins, Dipanjan Das, and Shashi Narayan. 2022. Query refinement prompts for closed-book long-form question answering. arXiv preprint arXiv:2210.17525. [↗](https://arxiv.org/abs/2210.17525)
2. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901. [find](https://scholar.google.com/scholar?q=Tom+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+et+al%2E+2020%2E+Language+models+are+few%2Dshot+lear)
3. Marie Duží and Martina Číhalová. 2015. Questions, answers, and presuppositions. Computación y Sistemas, 19(4):647–659. [↗](https://doi.org/10.13053/cys-19-4-2327)
4. Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli. 2019. Eli5: Long form question answering. arXiv preprint arXiv:1907.09190. [↗](https://arxiv.org/abs/1907.09190)
5. Liping Ge. 2011. Pragmatic functions of presupposition in advertising english. Asian Culture and History, 3(1):153. [↗](https://doi.org/10.5539/ach.v3n1p153)
6. Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. 2022. Decomposed prompting: A modular approach for solving complex tasks. arXiv preprint arXiv:2210.02406. [↗](https://arxiv.org/abs/2210.02406)
7. Najoung Kim, Phu Mon Htut, Samuel R Bowman, and Jackson Petty. 2022. Θ2: Question answering with questionable assumptions. arXiv preprint arXiv:2212.10003. [↗](https://arxiv.org/abs/2212.10003)
8. Najoung Kim, Ellie Pavlick, Burcu Karagol Ayan, and Deepak Ramachandran. 2021. Which linguist invented the lightbulb? presupposition verification for question-answering. arXiv preprint arXiv:2101.00391. [↗](https://arxiv.org/abs/2101.00391) [our card](https://md.mova.elamur.ai/free/en/docs/papers/kim-2021-which-linguist-invented-lightbulb-presupposition/)
9. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199– 22213. [↗](https://doi.org/10.52202/068431-1613)
10. Kalpesh Krishna, Aurko Roy, and Mohit Iyyer. 2021. Hurdles to progress in long-form question answering. arXiv preprint arXiv:2103.06332. [↗](https://arxiv.org/abs/2103.06332)
11. Anastasia Krithara, Anastasios Nentidis, Konstantinos Bougiatiotis, and Georgios Paliouras. 2023. Bioasq-qa: A manually curated corpus for biomedical question answering. Scientific Data, 10(1):170. [↗](https://doi.org/10.1038/s41597-023-02068-4)
12. Daliang Li, Ankit Singh Rawat, Manzil Zaheer, Xin Wang, Michal Lukasik, Andreas Veit, Felix Yu, and Sanjiv Kumar. 2022. Large language models with controllable working memory. arXiv preprint arXiv:2211.05110. [↗](https://arxiv.org/abs/2211.05110)
13. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81. [find](https://scholar.google.com/scholar?q=Chin%2DYew+Lin%2E+2004%2E+Rouge%3A+A+package+for+automatic+evaluation+of+summaries%2E+In+Text+summarization+branches+out%2C+pages+74%E2%80%9381%2E)
14. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. [↗](https://arxiv.org/abs/1907.11692)
15. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2023. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651. [↗](https://arxiv.org/abs/2303.17651)
16. Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906–1919, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2020.acl-main.173)
17. George A Miller, Galanter Eugene, and Karl H Pri-bram. 2017. Plans and the structure of behaviour. In Systems Research for Behavioral Sciencesystems Research, pages 369–382. Routledge. [find](https://scholar.google.com/scholar?q=George+A+Miller%2C+Galanter+Eugene%2C+and+Karl+H+Pri%2Dbram%2E+2017%2E+Plans+and+the+structure+of+behaviour%2E+In+Systems+Research+for+Behavioral+Sciencesystems+Research%2C+pages+369%E2%80%93382%2E+Routledge%2E)
18. Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020. Ambigqa: Answering ambiguous open-domain questions. arXiv preprint arXiv:2004.10645. [↗](https://arxiv.org/abs/2004.10645)
19. OpenAI. 2023. Gpt-4 technical report. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2023%2E+Gpt%2D4+technical+report%2E)
20. Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. 2022. Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.03350. [↗](https://arxiv.org/abs/2210.03350)
21. Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don’t know: Unanswerable questions for squad. arXiv preprint arXiv:1806.03822. [↗](https://arxiv.org/abs/1806.03822)
22. William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, and Jan Leike. 2022. Self-critiquing models for assisting human evaluators. arXiv preprint arXiv:2206.05802. [↗](https://arxiv.org/abs/2206.05802)
23. Thibault Sellam, Dipanjan Das, and Ankur P Parikh. 2020. Bleurt: Learning robust metrics for text generation. In Proceedings of ACL. [↗](https://doi.org/10.18653/v1/2020.acl-main.704)
24. Noah Shinn, Federico Cassano, Beck Labash, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. arXiv preprint arXiv:2303.11366, 14. [↗](https://arxiv.org/abs/2303.11366)
25. Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. 2022. Asqa: Factoid questions meet long-form answers. arXiv preprint arXiv:2204.06092. [↗](https://arxiv.org/abs/2204.06092)
26. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903. [↗](https://arxiv.org/abs/2201.11903)
27. Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Kang Liu, and Jun Zhao. 2023. Large language models are better reasoners with self-verification. CoRR, abs/2212.09561. [↗](https://arxiv.org/abs/2212.09561)
28. Fangyuan Xu, Yixiao Song, Mohit Iyyer, and Eunsol Choi. 2023. A critical evaluation of evaluations for long-form question answering. arXiv preprint arXiv:2305.18201. [↗](https://arxiv.org/abs/2305.18201)
29. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629. [↗](https://arxiv.org/abs/2210.03629)
30. Xinyan Velocity Yu, Sewon Min, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2022. Crepe: Open-domain question answering with false presuppositions. arXiv preprint arXiv:2211.17257. [↗](https://arxiv.org/abs/2211.17257) [our card](https://md.mova.elamur.ai/free/en/docs/papers/yu-2023-crepe-open-domain-question-answering/)
31. Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A Smith. 2023. How language model hallucinations can snowball. arXiv preprint arXiv:2305.13534. [↗](https://arxiv.org/abs/2305.13534)