[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/yang-2025-llm-reasoning-distracted-irrelevant-context/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark

# How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark

**Authors:** Minglai Yang, Ethan Huang, Liang Zhang, [Mihai Surdeanu](https://md.mova.elamur.ai/free/en/docs/papers/authors/mihai-surdeanu/), William Wang, Liangming Pan · **Year:** 2025 · **Venue:** arXiv preprint (arXiv:2505.18761) **Link:** [arxiv.org/abs/2505.18761](https://arxiv.org/abs/2505.18761) (DOI [10.48550/arXiv.2505.18761](https://doi.org/10.48550/arXiv.2505.18761)) **License of the paper:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) (arXiv) — [creativecommons.org/licenses/by/4.0](http://creativecommons.org/licenses/by/4.0/)

[**17** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/yang-2025-llm-reasoning-distracted-irrelevant-context/) of the 140673 papers whose bibliographies we readarXiv:2505.18761[licence](http://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://arxiv.org/abs/2505.18761)
- [PDF](https://arxiv.org/pdf/2505.18761)
- [arXiv 2505.18761](https://arxiv.org/abs/2505.18761)

**Authors (6)**

1. Minglai Yang
2. Ethan Huang
3. Liang Zhang
4. Mihai Surdeanu
5. William Wang
6. Liangming Pan

Citations and author metrics from OpenAlex (); authors with the same name are not always told apart.

## Summary

The paper measures how much large language models are thrown off in grade-school math reasoning when a problem contains irrelevant information. Earlier benchmarks such as GSM-IC added a single unrelated sentence without control over its structure. The authors build GSM-DC, where each problem is a directed acyclic graph of dependencies between quantities: the solution path is found by topological sorting, and nodes outside that path serve as controlled distractors before the graph is rendered into text with templates. This lets reasoning depth and amount of noise be varied independently and every reasoning step be checked automatically. Accuracy drops steadily as distractors increase, affecting both the choice of reasoning path and the arithmetic. Models trained on strong distractors were the most robust, and a stepwise tree search guided by a process reward model added further robustness out of distribution.

## How Mova uses it

- `math/src/bin/mathsolve.rs`, flag `--augment`: following the idea of training on noisy problems (together with Anantheswaran et al. 2024), each training problem gets a copy with a sentence containing a number from another problem inserted before the question; the target equation stays the same.
- The adaptation is much cruder than the paper: distractors are random sentences from other problems at a fixed position, not graph-controlled nodes related to the problem.

## Effectiveness in Mova

Measured and harmful: SVAMP 57.3% → 52.7% (same result in two runs). The artificial distractor always sits in the same place, so the model learns the position rather than the principle; in the papers distractors are tailored to the problem (similar object, same topic). The idea is not used by default.

---

👨‍🔬💥

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [arXiv 2505.18761](https://arxiv.org/abs/2505.18761)

```
@misc{yang2025how,
  title = {{How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark}},
  author = {Yang, Minglai and Huang, Ethan and Zhang, Liang and Surdeanu, Mihai and Wang, William and Pan, Liangming},
  year = {2025},
  eprint = {2505.18761},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.18761}
}
```

- [← previous Fantastic Questions and Where to Find Them: FairytaleQA – An Authentic Dataset for Narrative Comprehension](https://md.mova.elamur.ai/free/en/docs/papers/xu-2022-fantastic-questions-where-find-them/)
- [next → Assessing the influence of attractor-verb distance on grammatical agreement in humans and language models](https://md.mova.elamur.ai/free/en/docs/papers/zacharopoulos-2023-assessing-influence-attractor-verb-distance/)
- [↑ back to papers](https://md.mova.elamur.ai/free/en/docs/papers/)

## References

51 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Zeyuan Allen-Zhu and Yuanzhi Li. 2024a. Physics of language models: Part 1, learning hierarchical language structures. Preprint, arXiv:2305.13673. [find](https://scholar.google.com/scholar?q=Zeyuan+Allen%2DZhu+and+Yuanzhi+Li%2E+2024a%2E+Physics+of+language+models%3A+Part+1%2C+learning+hierarchical+language+structures%2E+Preprint%2C+arXiv%3A2305%2E13673%2E)
2. Zeyuan Allen-Zhu and Yuanzhi Li. 2024b. Physics of language models: Part 3.2, knowledge manipulation. Preprint, arXiv:2309.14402. [find](https://scholar.google.com/scholar?q=Zeyuan+Allen%2DZhu+and+Yuanzhi+Li%2E+2024b%2E+Physics+of+language+models%3A+Part+3%2E2%2C+knowledge+manipulation%2E+Preprint%2C+arXiv%3A2309%2E14402%2E)
3. Zeyuan Allen-Zhu and Yuanzhi Li. 2024c. Physics of language models: Part 3.3, knowledge capacity scaling laws. Preprint, arXiv:2404.05405. — [arXiv:2404.05405](https://arxiv.org/abs/2404.05405) [find](https://scholar.google.com/scholar?q=Zeyuan+Allen%2DZhu+and+Yuanzhi+Li%2E+2024c%2E+Physics+of+language+models%3A+Part+3%2E3%2C+knowledge+capacity+scaling+laws%2E+Preprint%2C+arXiv%3A2404%2E05405%2E+%E2%80%94+%5BarXiv%3A2404%2E05405%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2404%2E05405%29)
4. Ujjwala Anantheswaran, Himanshu Gupta, Kevin Scaria, Shreyas Verma, Chitta Baral, and Swaroop Mishra. 2024. Cutting through the noise: Boosting llm performance on math word problems. arXiv preprint arXiv:2406.15444. [find](https://scholar.google.com/scholar?q=Ujjwala+Anantheswaran%2C+Himanshu+Gupta%2C+Kevin+Scaria%2C+Shreyas+Verma%2C+Chitta+Baral%2C+and+Swaroop+Mishra%2E+2024%2E+Cutting+through+the+noise%3A+Boosting+llm+performance+on+math+word+problems%2E+arXiv+preprint+ar) [our card](https://md.mova.elamur.ai/free/en/docs/papers/anantheswaran-2024-cutting-through-noise-boosting-llm/)
5. Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans. 2024. The reversal curse: Llms trained on "a is b" fail to learn "b is a". Preprint, arXiv:2309.12288. [find](https://scholar.google.com/scholar?q=Lukas+Berglund%2C+Meg+Tong%2C+Max+Kaufmann%2C+Mikita+Balesni%2C+Asa+Cooper+Stickland%2C+Tomasz+Korbak%2C+and+Owain+Evans%2E+2024%2E+The+reversal+curse%3A+Llms+trained+on+%22a+is+b%22+fail+to+learn+%22b+is+a%22%2E+Preprint%2C+arXiv)
6. Leonardo Bertolazzi, Albert Gatt, and Raffaella Bernardi. 2024. A systematic analysis of large language models as soft reasoners: The case of syllogistic inferences. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16,… [find](https://scholar.google.com/scholar?q=Leonardo+Bertolazzi%2C+Albert+Gatt%2C+and+Raffaella+Bernardi%2E+2024%2E+A+systematic+analysis+of+large+language+models+as+soft+reasoners%3A+The+case+of+syllogistic+inferences%2E+In+Proceedings+of+the+2024+Confere)
7. Michael Boratko, Xiang Li, Tim O’Gorman, Rajarshi Das, Dan Le, and Andrew McCallum. 2020. ProtoQA: A question answering dataset for prototypical common-sense reasoning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1122–1136, Online. Associa… [find](https://scholar.google.com/scholar?q=Michael+Boratko%2C+Xiang+Li%2C+Tim+O%E2%80%99Gorman%2C+Rajarshi+Das%2C+Dan+Le%2C+and+Andrew+McCallum%2E+2020%2E+ProtoQA%3A+A+question+answering+dataset+for+prototypical+common%2Dsense+reasoning%2E+In+Proceedings+of+the+2020+Conf)
8. Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, and Jiaya Jia. 2024. Longlora: Efficient fine-tuning of long-context large language models. Preprint, arXiv:2309.12307. [find](https://scholar.google.com/scholar?q=Yukang+Chen%2C+Shengju+Qian%2C+Haotian+Tang%2C+Xin+Lai%2C+Zhijian+Liu%2C+Song+Han%2C+and+Jiaya+Jia%2E+2024%2E+Longlora%3A+Efficient+fine%2Dtuning+of+long%2Dcontext+large+language+models%2E+Preprint%2C+arXiv%3A2309%2E12307%2E)
9. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. [find](https://scholar.google.com/scholar?q=Karl+Cobbe%2C+Vineet+Kosaraju%2C+Mohammad+Bavarian%2C+Mark+Chen%2C+Heewoo+Jun%2C+Lukasz+Kaiser%2C+Matthias+Plappert%2C+Jerry+Tworek%2C+Jacob+Hilton%2C+Reiichiro+Nakano%2C+Christopher+Hesse%2C+and+John+Schulman%2E+2021%2E+Train)
10. Kuicai Dong, Yujing Chang, Xin Deik Goh, Dexun Li, Ruiming Tang, and Yong Liu. 2025a. Mmdocir: Benchmarking multi-modal retrieval for long documents. Preprint, arXiv:2501.08828. — [arXiv:2501.08828](https://arxiv.org/abs/2501.08828) [find](https://scholar.google.com/scholar?q=Kuicai+Dong%2C+Yujing+Chang%2C+Xin+Deik+Goh%2C+Dexun+Li%2C+Ruiming+Tang%2C+and+Yong+Liu%2E+2025a%2E+Mmdocir%3A+Benchmarking+multi%2Dmodal+retrieval+for+long+documents%2E+Preprint%2C+arXiv%3A2501%2E08828%2E+%E2%80%94+%5BarXiv%3A2501%2E08828%5D%28h)
11. Kuicai Dong, Yujing Chang, Shijie Huang, Yasheng Wang, Ruiming Tang, and Yong Liu. 2025b. Benchmarking retrieval-augmented multimomal generation for document question answering. Preprint, arXiv:2505.16470. — [arXiv:2505.16470](https://arxiv.org/abs/2505.16470) [find](https://scholar.google.com/scholar?q=Kuicai+Dong%2C+Yujing+Chang%2C+Shijie+Huang%2C+Yasheng+Wang%2C+Ruiming+Tang%2C+and+Yong+Liu%2E+2025b%2E+Benchmarking+retrieval%2Daugmented+multimomal+generation+for+document+question+answering%2E+Preprint%2C+arXiv%3A2505%2E1)
12. Kuicai Dong, Derrick Goh Xin Deik, Yi Quan Lee, Hao Zhang, Xiangyang Li, Cong Zhang, and Yong Liu. 2024. MC-indexing: Effective long document retrieval via multi-view content-aware indexing. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 2673–2691, Miami, Florida, US… [find](https://scholar.google.com/scholar?q=Kuicai+Dong%2C+Derrick+Goh+Xin+Deik%2C+Yi+Quan+Lee%2C+Hao+Zhang%2C+Xiangyang+Li%2C+Cong+Zhang%2C+and+Yong+Liu%2E+2024%2E+MC%2Dindexing%3A+Effective+long+document+retrieval+via+multi%2Dview+content%2Daware+indexing%2E+In+Findin)
13. Barbara A. Eriksen and Charles W. Eriksen. 1974. Effects of noise letters upon the identification of a target letter in a nonsearch task. Perception & Psychophysics, 16:143–149. [find](https://scholar.google.com/scholar?q=Barbara+A%2E+Eriksen+and+Charles+W%2E+Eriksen%2E+1974%2E+Effects+of+noise+letters+upon+the+identification+of+a+target+letter+in+a+nonsearch+task%2E+Perception+%26+Psychophysics%2C+16%3A143%E2%80%93149%2E)
14. Shibo Hao, Yi Gu, Haotian Luo, Tianyang Liu, Xiyan Shao, Xinyuan Wang, Shuhua Xie, Haodi Ma, Adithya Samavedhi, Qiyue Gao, Zhen Wang, and Zhiting Hu. 2024. LLM reasoners: New evaluation, library, and analysis of step-by-step reasoning with large language models. In First Conference on Language Model… [find](https://scholar.google.com/scholar?q=Shibo+Hao%2C+Yi+Gu%2C+Haotian+Luo%2C+Tianyang+Liu%2C+Xiyan+Shao%2C+Xinyuan+Wang%2C+Shuhua+Xie%2C+Haodi+Ma%2C+Adithya+Samavedhi%2C+Qiyue+Gao%2C+Zhen+Wang%2C+and+Zhiting+Hu%2E+2024%2E+LLM+reasoners%3A+New+evaluation%2C+library%2C+and+)
15. Arian Hosseini, Xingdi Yuan, Nikolay Malkin, Aaron C. Courville, Alessandro Sordoni, and Rishabh Agarwal. 2024. V-star: Training verifiers for self-taught reasoners. CoRR, abs/2402.06457. [find](https://scholar.google.com/scholar?q=Arian+Hosseini%2C+Xingdi+Yuan%2C+Nikolay+Malkin%2C+Aaron+C%2E+Courville%2C+Alessandro+Sordoni%2C+and+Rishabh+Agarwal%2E+2024%2E+V%2Dstar%3A+Training+verifiers+for+self%2Dtaught+reasoners%2E+CoRR%2C+abs%2F2402%2E06457%2E)
16. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2024. Large language models cannot self-correct reasoning yet. Preprint, arXiv:2310.01798. [find](https://scholar.google.com/scholar?q=Jie+Huang%2C+Xinyun+Chen%2C+Swaroop+Mishra%2C+Huaixiu+Steven+Zheng%2C+Adams+Wei+Yu%2C+Xinying+Song%2C+and+Denny+Zhou%2E+2024%2E+Large+language+models+cannot+self%2Dcorrect+reasoning+yet%2E+Preprint%2C+arXiv%3A2310%2E01798%2E)
17. Yining Huang, Bin Li, Keke Tang, and Meilian Chen. 2025. Lora-par: A flexible dual-system lora partitioning approach to efficient llm fine-tuning. Preprint, arXiv:2507.20999. [find](https://scholar.google.com/scholar?q=Yining+Huang%2C+Bin+Li%2C+Keke+Tang%2C+and+Meilian+Chen%2E+2025%2E+Lora%2Dpar%3A+A+flexible+dual%2Dsystem+lora+partitioning+approach+to+efficient+llm+fine%2Dtuning%2E+Preprint%2C+arXiv%3A2507%2E20999%2E)
18. Ming Jiang, Tingting Huang, Biao Guo, Yao Lu, and Feng Zhang. 2024a. Enhancing robustness in large language models: Prompting for mitigating the impact of irrelevant information. CoRR, abs/2408.10615. — [arXiv:2408.10615](https://arxiv.org/abs/2408.10615) [find](https://scholar.google.com/scholar?q=Ming+Jiang%2C+Tingting+Huang%2C+Biao+Guo%2C+Yao+Lu%2C+and+Feng+Zhang%2E+2024a%2E+Enhancing+robustness+in+large+language+models%3A+Prompting+for+mitigating+the+impact+of+irrelevant+information%2E+CoRR%2C+abs%2F2408%2E10615%2E)
19. Ziyan Jiang, Xueguang Ma, and Wenhu Chen. 2024b. Longrag: Enhancing retrieval-augmented generation with long-context llms. Preprint, arXiv:2406.15319. — [arXiv:2406.15319](https://arxiv.org/abs/2406.15319) [find](https://scholar.google.com/scholar?q=Ziyan+Jiang%2C+Xueguang+Ma%2C+and+Wenhu+Chen%2E+2024b%2E+Longrag%3A+Enhancing+retrieval%2Daugmented+generation+with+long%2Dcontext+llms%2E+Preprint%2C+arXiv%3A2406%2E15319%2E+%E2%80%94+%5BarXiv%3A2406%2E15319%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2406%2E1)
20. Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D. Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, Lei M. Zhang, Kay McKinney, Disha Shrivastava, Cosmin Paduraru, George Tucker, Doina Precup, Feryal M. P. Behbahani, and Aleksandra Faust. 2024. Training lang… [find](https://scholar.google.com/scholar?q=Aviral+Kumar%2C+Vincent+Zhuang%2C+Rishabh+Agarwal%2C+Yi+Su%2C+John+D%2E+Co%2DReyes%2C+Avi+Singh%2C+Kate+Baumli%2C+Shariq+Iqbal%2C+Colton+Bishop%2C+Rebecca+Roelofs%2C+Lei+M%2E+Zhang%2C+Kay+McKinney%2C+Disha+Shrivastava%2C+Cosmin+Padu)
21. Aitor Lewkowycz, Eric Zelikman, Ran Tao, Tatsunori Hashimoto, John Schulman, Xinyun Chen, C Lawrence Zitnick, Christopher D Manning, Daniel Saunders, Adam Santoro, et al. 2022. Solving quantitative reasoning problems with language models. arXiv preprint arXiv:2206.14858. [find](https://scholar.google.com/scholar?q=Aitor+Lewkowycz%2C+Eric+Zelikman%2C+Ran+Tao%2C+Tatsunori+Hashimoto%2C+John+Schulman%2C+Xinyun+Chen%2C+C+Lawrence+Zitnick%2C+Christopher+D+Manning%2C+Daniel+Saunders%2C+Adam+Santoro%2C+et+al%2E+2022%2E+Solving+quantitative+re)
22. Zhiming Li, Yushi Cao, Xiufeng Xu, Junzhe Jiang, Xu Liu, Yon Shin Teo, Shang-Wei Lin, and Yang Liu. 2024. Llms for relational reasoning: How far are we? In LLM4CODE@ICSE, pages 119–126. [find](https://scholar.google.com/scholar?q=Zhiming+Li%2C+Yushi+Cao%2C+Xiufeng+Xu%2C+Junzhe+Jiang%2C+Xu+Liu%2C+Yon+Shin+Teo%2C+Shang%2DWei+Lin%2C+and+Yang+Liu%2E+2024%2E+Llms+for+relational+reasoning%3A+How+far+are+we%3F+In+LLM4CODE%40ICSE%2C+pages+119%E2%80%93126%2E)
23. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. Let’s verify step by step. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. Open-Rev… [find](https://scholar.google.com/scholar?q=Hunter+Lightman%2C+Vineet+Kosaraju%2C+Yuri+Burda%2C+Harrison+Edwards%2C+Bowen+Baker%2C+Teddy+Lee%2C+Jan+Leike%2C+John+Schulman%2C+Ilya+Sutskever%2C+and+Karl+Cobbe%2E+2024%2E+Let%E2%80%99s+verify+step+by+step%2E+In+The+Twelfth+Intern)
24. Wanlong Liu, Junying Chen, Ke Ji, Li Zhou, Wenyu Chen, and Benyou Wang. 2024. Rag-instruct: Boosting llms with diverse retrieval-augmented instructions. Preprint, arXiv:2501.00353. [find](https://scholar.google.com/scholar?q=Wanlong+Liu%2C+Junying+Chen%2C+Ke+Ji%2C+Li+Zhou%2C+Wenyu+Chen%2C+and+Benyou+Wang%2E+2024%2E+Rag%2Dinstruct%3A+Boosting+llms+with+diverse+retrieval%2Daugmented+instructions%2E+Preprint%2C+arXiv%3A2501%2E00353%2E)
25. Bhargavi Paranjape, Scott Lundberg, Sameer Singh, Hannaneh Hajishirzi, Luke Zettlemoyer, and Marco Tulio Ribeiro. 2023. Art: Automatic multi-step reasoning and tool-use for large language models. Preprint, arXiv:2303.09014. [find](https://scholar.google.com/scholar?q=Bhargavi+Paranjape%2C+Scott+Lundberg%2C+Sameer+Singh%2C+Hannaneh+Hajishirzi%2C+Luke+Zettlemoyer%2C+and+Marco+Tulio+Ribeiro%2E+2023%2E+Art%3A+Automatic+multi%2Dstep+reasoning+and+tool%2Duse+for+large+language+models%2E+Prep)
26. Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. In Thirty-seventh Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Timo+Schick%2C+Jane+Dwivedi%2DYu%2C+Roberto+Dessi%2C+Roberta+Raileanu%2C+Maria+Lomeli%2C+Eric+Hambro%2C+Luke+Zettlemoyer%2C+Nicola+Cancedda%2C+and+Thomas+Scialom%2E+2023%2E+Toolformer%3A+Language+models+can+teach+themselves+)
27. Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Schärli, and Denny Zhou. 2023a. Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 20… [find](https://scholar.google.com/scholar?q=Freda+Shi%2C+Xinyun+Chen%2C+Kanishka+Misra%2C+Nathan+Scales%2C+David+Dohan%2C+Ed+H%2E+Chi%2C+Nathanael+Sch%C3%A4rli%2C+and+Denny+Zhou%2E+2023a%2E+Large+language+models+can+be+easily+distracted+by+irrelevant+context%2E+In+Intern)
28. Xiaoming Shi, Siqiao Xue, Kangrui Wang, Fan Zhou, James Y. Zhang, Jun Zhou, Chenhao Tan, and Hongyuan Mei. 2023b. Language models can improve event prediction by few-shot abductive reasoning. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing… [find](https://scholar.google.com/scholar?q=Xiaoming+Shi%2C+Siqiao+Xue%2C+Kangrui+Wang%2C+Fan+Zhou%2C+James+Y%2E+Zhang%2C+Jun+Zhou%2C+Chenhao+Tan%2C+and+Hongyuan+Mei%2E+2023b%2E+Language+models+can+improve+event+prediction+by+few%2Dshot+abductive+reasoning%2E+In+Advan)
29. Seok Hwan Song and Wallapak Tavanapong. 2024. How much do prompting methods help llms on quantitative reasoning with irrelevant information? In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, CIKM 2024, Boise, ID, USA, October 21-25, 2024, pages 2128–213… [find](https://scholar.google.com/scholar?q=Seok+Hwan+Song+and+Wallapak+Tavanapong%2E+2024%2E+How+much+do+prompting+methods+help+llms+on+quantitative+reasoning+with+irrelevant+information%3F+In+Proceedings+of+the+33rd+ACM+International+Conference+on+)
30. Oyvind Tafjord, Bhavana Dalvi Mishra, and Peter Clark. 2021. Proofwriter: Generating implications, proofs, and abductive statements over natural language. Preprint, arXiv:2012.13048. [find](https://scholar.google.com/scholar?q=Oyvind+Tafjord%2C+Bhavana+Dalvi+Mishra%2C+and+Peter+Clark%2E+2021%2E+Proofwriter%3A+Generating+implications%2C+proofs%2C+and+abductive+statements+over+natural+language%2E+Preprint%2C+arXiv%3A2012%2E13048%2E)
31. Shi-Yu Tian, Zhi Zhou, Kun-Yang Yu, Ming Yang, Lin-Han Jia, Lan-Zhe Guo, and Yu-Feng Li. 2025. Vc search: Bridging the gap between well-defined and ill-defined problems in mathematical reasoning. Preprint, arXiv:2406.05055. [find](https://scholar.google.com/scholar?q=Shi%2DYu+Tian%2C+Zhi+Zhou%2C+Kun%2DYang+Yu%2C+Ming+Yang%2C+Lin%2DHan+Jia%2C+Lan%2DZhe+Guo%2C+and+Yu%2DFeng+Li%2E+2025%2E+Vc+search%3A+Bridging+the+gap+between+well%2Ddefined+and+ill%2Ddefined+problems+in+mathematical+reasoning%2E+Prep)
32. Jonathan Uesato, Nate Kushman, Ramana Kumar, H. Francis Song, Noah Y. Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. 2022. Solving math word problems with process- and outcome-based feedback. CoRR, abs/2211.14275. [find](https://scholar.google.com/scholar?q=Jonathan+Uesato%2C+Nate+Kushman%2C+Ramana+Kumar%2C+H%2E+Francis+Song%2C+Noah+Y%2E+Siegel%2C+Lisa+Wang%2C+Antonia+Creswell%2C+Geoffrey+Irving%2C+and+Irina+Higgins%2E+2022%2E+Solving+math+word+problems+with+process%2D+and+outcom)
33. Xiaonan Wang, Bo Shao, and Hansaem Kim. 2025a. Toward reliable vlm: A fine-grained benchmark and framework for exposure, bias, and inference in korean street views. Preprint, arXiv:2506.03371. — [arXiv:2506.03371](https://arxiv.org/abs/2506.03371) [find](https://scholar.google.com/scholar?q=Xiaonan+Wang%2C+Bo+Shao%2C+and+Hansaem+Kim%2E+2025a%2E+Toward+reliable+vlm%3A+A+fine%2Dgrained+benchmark+and+framework+for+exposure%2C+bias%2C+and+inference+in+korean+street+views%2E+Preprint%2C+arXiv%3A2506%2E03371%2E+%E2%80%94+%5BarXi)
34. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5,… [find](https://scholar.google.com/scholar?q=Xuezhi+Wang%2C+Jason+Wei%2C+Dale+Schuurmans%2C+Quoc+V%2E+Le%2C+Ed+H%2E+Chi%2C+Sharan+Narang%2C+Aakanksha+Chowdhery%2C+and+Denny+Zhou%2E+2023%2E+Self%2Dconsistency+improves+chain+of+thought+reasoning+in+language+models%2E+In+Th)
35. Yifei Wang, Feng Xiong, Yong Wang, Linjing Li, Xiangxiang Chu, and Daniel Dajun Zeng. 2025b. Position bias mitigates position bias:mitigate position bias through inter-position knowledge distillation. Preprint, arXiv:2508.15709. [find](https://scholar.google.com/scholar?q=Yifei+Wang%2C+Feng+Xiong%2C+Yong+Wang%2C+Linjing+Li%2C+Xiangxiang+Chu%2C+and+Daniel+Dajun+Zeng%2E+2025b%2E+Position+bias+mitigates+position+bias%3Amitigate+position+bias+through+inter%2Dposition+knowledge+distillation%2E)
36. Jiyao Wei, Saiping Guan, Da Li, Xiaolong Jin, Jiafeng Guo, and Xueqi Cheng. 2025. A survey of link prediction in n-ary knowledge graphs. Preprint, arXiv:2506.08970. [find](https://scholar.google.com/scholar?q=Jiyao+Wei%2C+Saiping+Guan%2C+Da+Li%2C+Xiaolong+Jin%2C+Jiafeng+Guo%2C+and+Xueqi+Cheng%2E+2025%2E+A+survey+of+link+prediction+in+n%2Dary+knowledge+graphs%2E+Preprint%2C+arXiv%3A2506%2E08970%2E)
37. Siye Wu, Jian Xie, Jiangjie Chen, Tinghui Zhu, Kai Zhang, and Yanghua Xiao. 2024. How easily do irrelevant inputs skew the responses of large language models? CoRR, abs/2404.03302. [find](https://scholar.google.com/scholar?q=Siye+Wu%2C+Jian+Xie%2C+Jiangjie+Chen%2C+Tinghui+Zhu%2C+Kai+Zhang%2C+and+Yanghua+Xiao%2E+2024%2E+How+easily+do+irrelevant+inputs+skew+the+responses+of+large+language+models%3F+CoRR%2C+abs%2F2404%2E03302%2E)
38. Zikai Xiao, Ziyang Wang, Wen Ma, Yan Zhang, Wei Shen, WangYan WangYan, Luqi Gong, and Zuozhu Liu. 2025. Mitigating posterior salience attenuation in long-context LLMs with positional contrastive decoding. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Vol… [find](https://scholar.google.com/scholar?q=Zikai+Xiao%2C+Ziyang+Wang%2C+Wen+Ma%2C+Yan+Zhang%2C+Wei+Shen%2C+WangYan+WangYan%2C+Luqi+Gong%2C+and+Zuozhu+Liu%2E+2025%2E+Mitigating+posterior+salience+attenuation+in+long%2Dcontext+LLMs+with+positional+contrastive+decod)
39. Feng Xiong, Hongling Xu, Yifei Wang, Runxi Cheng, Yong Wang, and Xiangxiang Chu. 2025. Hs-star: Hierarchical sampling for self-taught reasoners via difficulty estimation and budget reallocation. Preprint, arXiv:2505.19866. [find](https://scholar.google.com/scholar?q=Feng+Xiong%2C+Hongling+Xu%2C+Yifei+Wang%2C+Runxi+Cheng%2C+Yong+Wang%2C+and+Xiangxiang+Chu%2E+2025%2E+Hs%2Dstar%3A+Hierarchical+sampling+for+self%2Dtaught+reasoners+via+difficulty+estimation+and+budget+reallocation%2E+Prepr)
40. Wenda Xu, Guanglei Zhu, Xuandong Zhao, Liangming Pan, Lei Li, and William Yang Wang. 2024. Pride and prejudice: Llm amplifies self-bias in self-refinement. Preprint, arXiv:2402.11436. [find](https://scholar.google.com/scholar?q=Wenda+Xu%2C+Guanglei+Zhu%2C+Xuandong+Zhao%2C+Liangming+Pan%2C+Lei+Li%2C+and+William+Yang+Wang%2E+2024%2E+Pride+and+prejudice%3A+Llm+amplifies+self%2Dbias+in+self%2Drefinement%2E+Preprint%2C+arXiv%3A2402%2E11436%2E)
41. Yuhui Xu, Lingxi Xie, Xiaotao Gu, Xin Chen, Heng Chang, Hengheng Zhang, Zhengsu Chen, Xiaopeng Zhang, and Qi Tian. 2023. Qa-lora: Quantization-aware low-rank adaptation of large language models. Preprint, arXiv:2309.14717. — [arXiv:2309.14717](https://arxiv.org/abs/2309.14717) [find](https://scholar.google.com/scholar?q=Yuhui+Xu%2C+Lingxi+Xie%2C+Xiaotao+Gu%2C+Xin+Chen%2C+Heng+Chang%2C+Hengheng+Zhang%2C+Zhengsu+Chen%2C+Xiaopeng+Zhang%2C+and+Qi+Tian%2E+2023%2E+Qa%2Dlora%3A+Quantization%2Daware+low%2Drank+adaptation+of+large+language+models%2E+Prepr)
42. Bo Yang, Qingping Yang, Yingwei Ma, and Runtao Liu. 2025. Utmath: Math evaluation with unit test via reasoning-to-coding thoughts. Preprint, arXiv:2411.07240. — [arXiv:2411.07240](https://arxiv.org/abs/2411.07240) [find](https://scholar.google.com/scholar?q=Bo+Yang%2C+Qingping+Yang%2C+Yingwei+Ma%2C+and+Runtao+Liu%2E+2025%2E+Utmath%3A+Math+evaluation+with+unit+test+via+reasoning%2Dto%2Dcoding+thoughts%2E+Preprint%2C+arXiv%3A2411%2E07240%2E+%E2%80%94+%5BarXiv%3A2411%2E07240%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
43. Shinn Yao, Jeffrey Zhao, Dian Yu, Izhak Zhao, Yujia Yu, Eric Gao, Maarten Bosma, Yi Wang, and Denny Zhou. 2023a. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601. [find](https://scholar.google.com/scholar?q=Shinn+Yao%2C+Jeffrey+Zhao%2C+Dian+Yu%2C+Izhak+Zhao%2C+Yujia+Yu%2C+Eric+Gao%2C+Maarten+Bosma%2C+Yi+Wang%2C+and+Denny+Zhou%2E+2023a%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E+arXiv+preprint)
44. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023b. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023,… [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Tom+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+2023b%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E+In+Advances+in+Neural)
45. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. 2023c. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Jeffrey+Zhao%2C+Dian+Yu%2C+Nan+Du%2C+Izhak+Shafran%2C+Karthik+R%2E+Narasimhan%2C+and+Yuan+Cao%2E+2023c%2E+React%3A+Synergizing+reasoning+and+acting+in+language+models%2E+In+The+Eleventh+International+Conferen)
46. Tian Ye, Zicheng Xu, Yuanzhi Li, and Zeyuan Allen-Zhu. 2025. Physics of language models: Part 2.1, grade-school math and the hidden reasoning process. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Tian+Ye%2C+Zicheng+Xu%2C+Yuanzhi+Li%2C+and+Zeyuan+Allen%2DZhu%2E+2025%2E+Physics+of+language+models%3A+Part+2%2E1%2C+grade%2Dschool+math+and+the+hidden+reasoning+process%2E+In+The+Thirteenth+International+Conference+on+Lea)
47. Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. 2022. Star: Bootstrapping reasoning with reasoning. Preprint, arXiv:2203.14465. [find](https://scholar.google.com/scholar?q=Eric+Zelikman%2C+Yuhuai+Wu%2C+Jesse+Mu%2C+and+Noah+D%2E+Goodman%2E+2022%2E+Star%3A+Bootstrapping+reasoning+with+reasoning%2E+Preprint%2C+arXiv%3A2203%2E14465%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/zelikman-2022-star-bootstrapping-reasoning-reasoning/)
48. Chujie Zheng, Zhenru Zhang, Beichen Zhang, Runji Lin, Keming Lu, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. 2024. Processbench: Identifying process errors in mathematical reasoning. CoRR, abs/2412.06559. [find](https://scholar.google.com/scholar?q=Chujie+Zheng%2C+Zhenru+Zhang%2C+Beichen+Zhang%2C+Runji+Lin%2C+Keming+Lu%2C+Bowen+Yu%2C+Dayiheng+Liu%2C+Jingren+Zhou%2C+and+Junyang+Lin%2E+2024%2E+Processbench%3A+Identifying+process+errors+in+mathematical+reasoning%2E+CoRR%2C+)
49. Xuezhi Zhou, Nathanael Schärli, Luheng Hou, Jason Wei, Swaroop Mishra, Xinyun Huang, Quoc Le, Hyung Won Chung, Hieu Pham, Barret Zoph, et al. 2022. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625. [find](https://scholar.google.com/scholar?q=Xuezhi+Zhou%2C+Nathanael+Sch%C3%A4rli%2C+Luheng+Hou%2C+Jason+Wei%2C+Swaroop+Mishra%2C+Xinyun+Huang%2C+Quoc+Le%2C+Hyung+Won+Chung%2C+Hieu+Pham%2C+Barret+Zoph%2C+et+al%2E+2022%2E+Least%2Dto%2Dmost+prompting+enables+complex+reasoning+in)
50. Yang Zhou, Hongyi Liu, Zhuoming Chen, Yuandong Tian, and Beidi Chen. 2025. Gsm-infinite: How do your llms behave over infinitely increasing context length and reasoning complexity? CoRR, abs/2502.05252. [find](https://scholar.google.com/scholar?q=Yang+Zhou%2C+Hongyi+Liu%2C+Zhuoming+Chen%2C+Yuandong+Tian%2C+and+Beidi+Chen%2E+2025%2E+Gsm%2Dinfinite%3A+How+do+your+llms+behave+over+infinitely+increasing+context+length+and+reasoning+complexity%3F+CoRR%2C+abs%2F2502%2E0525)
51. Content of Appendix A Dataset Samples A.1 Training Dataset with Different Level of IC for Finetuned Model A.2 Prompts for Closed-Sourced Models B Empirical Noise Stratification B Clarifying the Role of Synthetic Problem Generation D Operation-Range Bias and Our Train/Test Protocol E Process Reward M… [find](https://scholar.google.com/scholar?q=Content+of+Appendix+A+Dataset+Samples+A%2E1+Training+Dataset+with+Different+Level+of+IC+for+Finetuned+Model+A%2E2+Prompts+for+Closed%2DSourced+Models+B+Empirical+Noise+Stratification+B+Clarifying+the+Role+o)