[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/jia-2017-adversarial-examples-evaluating-reading-comprehension/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / Adversarial Examples for Evaluating Reading Comprehension Systems

# Adversarial Examples for Evaluating Reading Comprehension Systems

**Authors:** [Robin Jia](https://md.mova.elamur.ai/free/en/docs/papers/authors/robin-jia/), [Percy Liang](https://md.mova.elamur.ai/free/en/docs/papers/authors/percy-liang/) · **Year:** 2017 · **Venue:** EMNLP 2017 **Link:** [aclanthology.org/D17-1215](https://aclanthology.org/D17-1215/) · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)

[**1370** citations](https://openalex.org/works?filter=cites:W2963969878) openalex · 2026-10-18[**579** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/jia-2017-adversarial-examples-evaluating-reading-comprehension/) of the 140673 papers whose bibliographies we read[Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing](https://aclanthology.org/volumes/D17-1/)[licence](https://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://aclanthology.org/D17-1215/)
- [PDF](https://aclanthology.org/D17-1215.pdf)
- [DOI 10.18653/v1/d17-1215](https://doi.org/10.18653/v1/d17-1215)
- [arXiv 1707.07328](https://arxiv.org/abs/1707.07328)
- [OpenAlex](https://openalex.org/W2963969878)

**Authors (2)**

1. [Robin Jia](https://openalex.org/A5041906762) [ORCID](https://orcid.org/0009-0002-8123-7132) · h 19 · 6500 citations · 141 works
2. [Percy Liang](https://openalex.org/A5025255782) [ORCID](https://orcid.org/0000-0002-0458-6139) · h 91 · 49602 citations · 462 works

Citations and author metrics from openalex (2026-10-18); authors with the same name are not always told apart.

## What it is about

To test whether SQuAD readers understand or only match words, the authors add one distracting sentence to the paragraph that shares many words with the question but does not answer it. AddSent builds it by changing the question's content words (nouns and adjectives to nearby words, named entities and numbers to others), turning it into a statement with a fake answer of the right type, and having crowd workers fix grammar; the worst of up to five such sentences is used. AddOneSent adds a random approved one (model-independent); AddAny appends ten arbitrary words chosen by search. Average F1 of sixteen published models falls from 75% to 36% under AddSent, and to 6.7% under AddAny for four models, while humans lose only 3.4 points under AddOneSent. In 96.6% of failures the model answers from the added sentence; models survive more when the question shares a long n-gram with the passage. Training on generated distractors protects against the same generator but not against a variant that uses other fake answers and puts the sentence first.

## Ideas

- I1. An AddSent-style distractor gate for the SLM student, fully deterministic: for a course question, take the lesson sentence that answers it, replace the entity in the questioned role by another entity of the same type from the lesson (or a fixed list), change one other content word of the question (verb or a modifier), and insert the sentence into the lesson. The right answer must not change. Because the student matches roles in frames, it should be far more stable than word matchers; measure the share of answers that stay right, per course.
- I2. Run the distractor both appended and prepended, as AddSentMod does: a difference shows a position or recency bias in how the student picks among supporting frames (salience, last-mentioned), which should be explainable by the trace or removed.
- I3. Use the failures as a list of where role matching is too loose: a distractor accepted means a frame that differs from the question in one role still counted as support. This feeds item 1 of docs/mmm-diagnosis.md «Next» (penalise unsupported or contradicting content) with concrete cases.

## For Mova

[30. Pragmatics and discourse: overview](https://md.mova.elamur.ai/free/en/docs/languages/en/30-pragmatics-overview/) §9 (measurements). Code: `world/src/knowledge.rs` (support scoring), `world/src/report.rs`. Measures: share of answers unchanged under distractors (append/prepend), per course; course tests unchanged without distractors (the gate must not cost accuracy).

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.18653/v1/d17-1215](https://doi.org/10.18653/v1/d17-1215) · [arXiv 1707.07328](https://arxiv.org/abs/1707.07328)

```
@inproceedings{jia2017adversarial,
  title = {{Adversarial Examples for Evaluating Reading Comprehension Systems}},
  author = {Jia, Robin and Liang, Percy},
  booktitle = {Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  year = {2017},
  doi = {10.18653/v1/d17-1215},
  eprint = {1707.07328},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d17-1215}
}
```

- [← previous Negation in English and Other Languages](https://md.mova.elamur.ai/free/en/docs/papers/jespersen-1917-negation-english-other-languages/)
- [next → He Thinks He Knows Better than the Doctors: BERT for Event Factuality Fails on Pragmatics](https://md.mova.elamur.ai/free/en/docs/papers/jiang-2021-he-thinks-he-knows-better/)
- [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/)

## References

35 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. E. M. Bender, H. Daume III, A. Ettinger, H. Kannan, S. Rao, and E. Rothschild. 2017. Build it, break it: The language edition. https://bibinlp. umiacs.umd.edu/. [find](https://scholar.google.com/scholar?q=E%2E+M%2E+Bender%2C+H%2E+Daume+III%2C+A%2E+Ettinger%2C+H%2E+Kannan%2C+S%2E+Rao%2C+and+E%2E+Rothschild%2E+2017%2E+Build+it%2C+break+it%3A+The+language+edition%2E+https%3A%2F%2Fbibinlp%2E+umiacs%2Eumd%2Eedu%2F%2E)
2. S. R. Bowman, L. Vilnis, O. Vinyals, A. M. Dai, R. Jozefowicz, and S. Bengio. 2016. Generating sentences from a continuous space. In Computational Natural Language Learning (CoNLL). [find](https://scholar.google.com/scholar?q=S%2E+R%2E+Bowman%2C+L%2E+Vilnis%2C+O%2E+Vinyals%2C+A%2E+M%2E+Dai%2C+R%2E+Jozefowicz%2C+and+S%2E+Bengio%2E+2016%2E+Generating+sentences+from+a+continuous+space%2E+In+Computational+Natural+Language+Learning+%28CoNLL%29%2E)
3. N. Dalvi, P. Domingos, Mausam, S. Sanghai, and D. Verma. 2004. Adversarial classification. In International Conference on Knowledge Discovery and Data Mining (KDD). [find](https://scholar.google.com/scholar?q=N%2E+Dalvi%2C+P%2E+Domingos%2C+Mausam%2C+S%2E+Sanghai%2C+and+D%2E+Verma%2E+2004%2E+Adversarial+classification%2E+In+International+Conference+on+Knowledge+Discovery+and+Data+Mining+%28KDD%29%2E)
4. C. Fellbaum. 1998. WordNet: An Electronic Lexical Database. MIT Press. [find](https://scholar.google.com/scholar?q=C%2E+Fellbaum%2E+1998%2E+WordNet%3A+An+Electronic+Lexical+Database%2E+MIT+Press%2E)
5. W. N. Francis and H. Kucera. 1979. Brown Corpus Manual. [find](https://scholar.google.com/scholar?q=W%2E+N%2E+Francis+and+H%2E+Kucera%2E+1979%2E+Brown+Corpus+Manual%2E)
6. A. Globerson and S. Roweis. 2006. Nightmare at test time: robust learning by feature deletion. In International Conference on Machine Learning (ICML), pages 353–360. [find](https://scholar.google.com/scholar?q=A%2E+Globerson+and+S%2E+Roweis%2E+2006%2E+Nightmare+at+test+time%3A+robust+learning+by+feature+deletion%2E+In+International+Conference+on+Machine+Learning+%28ICML%29%2C+pages+353%E2%80%93360%2E)
7. Y. Gong and S. R. Bowman. 2017. Ruminating reader: Reasoning with gated multi-hop attention. arXiv. [find](https://scholar.google.com/scholar?q=Y%2E+Gong+and+S%2E+R%2E+Bowman%2E+2017%2E+Ruminating+reader%3A+Reasoning+with+gated+multi%2Dhop+attention%2E+arXiv%2E)
8. I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems (NIPS). [find](https://scholar.google.com/scholar?q=I%2E+J%2E+Goodfellow%2C+J%2E+Pouget%2DAbadie%2C+M%2E+Mirza%2C+B%2E+Xu%2C+D%2E+Warde%2DFarley%2C+S%2E+Ozair%2C+A%2E+Courville%2C+and+Y%2E+Bengio%2E+2014%2E+Generative+adversarial+nets%2E+In+Advances+in+Neural+Information+Processing+Systems+%28NI)
9. I. J. Goodfellow, J. Shlens, and C. Szegedy. 2015. Explaining and harnessing adversarial examples. In International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=I%2E+J%2E+Goodfellow%2C+J%2E+Shlens%2C+and+C%2E+Szegedy%2E+2015%2E+Explaining+and+harnessing+adversarial+examples%2E+In+International+Conference+on+Learning+Representations+%28ICLR%29%2E)
10. M. Hu, Y. Peng, and X. Qiu. 2017. Mnemonic reader for machine comprehension. arXiv. [find](https://scholar.google.com/scholar?q=M%2E+Hu%2C+Y%2E+Peng%2C+and+X%2E+Qiu%2E+2017%2E+Mnemonic+reader+for+machine+comprehension%2E+arXiv%2E)
11. R. Jia and P. Liang. 2016. Data recombination for neural semantic parsing. In Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=R%2E+Jia+and+P%2E+Liang%2E+2016%2E+Data+recombination+for+neural+semantic+parsing%2E+In+Association+for+Computational+Linguistics+%28ACL%29%2E)
12. K. Lee, S. Salant, T. Kwiatkowski, A. Parikh, D. Das, and J. Berant. 2017. Learning recurrent span representations for extractive question answering. arXiv. [find](https://scholar.google.com/scholar?q=K%2E+Lee%2C+S%2E+Salant%2C+T%2E+Kwiatkowski%2C+A%2E+Parikh%2C+D%2E+Das%2C+and+J%2E+Berant%2E+2017%2E+Learning+recurrent+span+representations+for+extractive+question+answering%2E+arXiv%2E)
13. H. J. Levesque. 2013. On our best behaviour. In International Joint Conference on Artificial Intelligence (IJCAI). [find](https://scholar.google.com/scholar?q=H%2E+J%2E+Levesque%2E+2013%2E+On+our+best+behaviour%2E+In+International+Joint+Conference+on+Artificial+Intelligence+%28IJCAI%29%2E)
14. C. D. Manning, M. Surdeanu, J. Bauer, J. Finkel, S. J. Bethard, and D. McClosky. 2014. The stanford coreNLP natural language processing toolkit. In ACL system demonstrations. [find](https://scholar.google.com/scholar?q=C%2E+D%2E+Manning%2C+M%2E+Surdeanu%2C+J%2E+Bauer%2C+J%2E+Finkel%2C+S%2E+J%2E+Bethard%2C+and+D%2E+McClosky%2E+2014%2E+The+stanford+coreNLP+natural+language+processing+toolkit%2E+In+ACL+system+demonstrations%2E)
15. M. Marcus, B. Santorini, M. A. Marcinkiewicz, and A. Taylor. 1999. Treebank-3. [find](https://scholar.google.com/scholar?q=M%2E+Marcus%2C+B%2E+Santorini%2C+M%2E+A%2E+Marcinkiewicz%2C+and+A%2E+Taylor%2E+1999%2E+Treebank%2D3%2E)
16. S. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, and P. Frossard. 2017. Universal adversarial perturbations. In Computer Vision and Pattern Recognition (CVPR). [find](https://scholar.google.com/scholar?q=S%2E+Moosavi%2DDezfooli%2C+A%2E+Fawzi%2C+O%2E+Fawzi%2C+and+P%2E+Frossard%2E+2017%2E+Universal+adversarial+perturbations%2E+In+Computer+Vision+and+Pattern+Recognition+%28CVPR%29%2E)
17. N. Narodytska and S. P. Kasiviswanathan. 2016. Simple black-box adversarial perturbations for deep networks. arXiv preprint arXiv:1612.06299. — [arXiv:1612.06299](https://arxiv.org/abs/1612.06299) [find](https://scholar.google.com/scholar?q=N%2E+Narodytska+and+S%2E+P%2E+Kasiviswanathan%2E+2016%2E+Simple+black%2Dbox+adversarial+perturbations+for+deep+networks%2E+arXiv+preprint+arXiv%3A1612%2E06299%2E+%E2%80%94+%5BarXiv%3A1612%2E06299%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1612%2E06299%29)
18. D. Paperno, G. Kruszewski, A. Lazaridou, Q. N. Pham, R. Bernardi, S. Pezzelle, M. Baroni, G. Boleda, and R. Fernandez. 2016. The LAMBADA dataset: Word prediction requiring a broad discourse context. In Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=D%2E+Paperno%2C+G%2E+Kruszewski%2C+A%2E+Lazaridou%2C+Q%2E+N%2E+Pham%2C+R%2E+Bernardi%2C+S%2E+Pezzelle%2C+M%2E+Baroni%2C+G%2E+Boleda%2C+and+R%2E+Fernandez%2E+2016%2E+The+LAMBADA+dataset%3A+Word+prediction+requiring+a+broad+discourse+context%2E+I)
19. N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. Celik, and A. Swami. 2017. Practical black-box attacks against deep learning systems using adversarial examples. In Proceedings of the ACM Asia Conference on Computer and Communications Security. [find](https://scholar.google.com/scholar?q=N%2E+Papernot%2C+P%2E+McDaniel%2C+I%2E+Goodfellow%2C+S%2E+Jha%2C+Z%2E+Celik%2C+and+A%2E+Swami%2E+2017%2E+Practical+black%2Dbox+attacks+against+deep+learning+systems+using+adversarial+examples%2E+In+Proceedings+of+the+ACM+Asia+Conf)
20. J. Pennington, R. Socher, and C. D. Manning. 2014. Glove: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=J%2E+Pennington%2C+R%2E+Socher%2C+and+C%2E+D%2E+Manning%2E+2014%2E+Glove%3A+Global+vectors+for+word+representation%2E+In+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2E)
21. P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=P%2E+Rajpurkar%2C+J%2E+Zhang%2C+K%2E+Lopyrev%2C+and+P%2E+Liang%2E+2016%2E+Squad%3A+100%2C000%2B+questions+for+machine+comprehension+of+text%2E+In+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2E)
22. L. Rimell, S. Clark, and M. Steedman. 2009. Unbounded dependency recovery for parser evaluation. In Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=L%2E+Rimell%2C+S%2E+Clark%2C+and+M%2E+Steedman%2E+2009%2E+Unbounded+dependency+recovery+for+parser+evaluation%2E+In+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2009-rimell-unbounded-dependency-recovery-parser-evaluation/)
23. M. Seo, A. Kembhavi, A. Farhadi, and H. Hajishirzi. 2016. Bidirectional attention flow for machine comprehension. arXiv. [find](https://scholar.google.com/scholar?q=M%2E+Seo%2C+A%2E+Kembhavi%2C+A%2E+Farhadi%2C+and+H%2E+Hajishirzi%2E+2016%2E+Bidirectional+attention+flow+for+machine+comprehension%2E+arXiv%2E)
24. J. Li, W. Monroe, T. Shi, A. Ritter, and D. Jurafsky. 2017. Adversarial learning for neural dialogue generation. arXiv preprint arXiv:1701.06547. [find](https://scholar.google.com/scholar?q=J%2E+Li%2C+W%2E+Monroe%2C+T%2E+Shi%2C+A%2E+Ritter%2C+and+D%2E+Jurafsky%2E+2017%2E+Adversarial+learning+for+neural+dialogue+generation%2E+arXiv+preprint+arXiv%3A1701%2E06547%2E)
25. Y. Shen, P. Huang, J. Gao, and W. Chen. 2017. Reasonet: Learning to stop reading in machine comprehension. In International Conference on Knowledge Discovery and Data Mining (KDD). [find](https://scholar.google.com/scholar?q=Y%2E+Shen%2C+P%2E+Huang%2C+J%2E+Gao%2C+and+W%2E+Chen%2E+2017%2E+Reasonet%3A+Learning+to+stop+reading+in+machine+comprehension%2E+In+International+Conference+on+Knowledge+Discovery+and+Data+Mining+%28KDD%29%2E)
26. R. Liu, J. Hu, W. Wei, Z. Yang, and E. Nyberg. 2017. Structural embedding of syntactic trees for machine comprehension. arXiv. [find](https://scholar.google.com/scholar?q=R%2E+Liu%2C+J%2E+Hu%2C+W%2E+Wei%2C+Z%2E+Yang%2C+and+E%2E+Nyberg%2E+2017%2E+Structural+embedding+of+syntactic+trees+for+machine+comprehension%2E+arXiv%2E)
27. N. A. Smith. 2012. Adversarial evaluation for models of natural language. arXiv preprint arXiv:1207.0245. — [arXiv:1207.0245](https://arxiv.org/abs/1207.0245) [find](https://scholar.google.com/scholar?q=N%2E+A%2E+Smith%2E+2012%2E+Adversarial+evaluation+for+models+of+natural+language%2E+arXiv+preprint+arXiv%3A1207%2E0245%2E+%E2%80%94+%5BarXiv%3A1207%2E0245%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1207%2E0245%29)
28. D. Lowd and C. Meek. 2005. Adversarial learning. In International Conference on Knowledge Discovery and Data Mining (KDD). [find](https://scholar.google.com/scholar?q=D%2E+Lowd+and+C%2E+Meek%2E+2005%2E+Adversarial+learning%2E+In+International+Conference+on+Knowledge+Discovery+and+Data+Mining+%28KDD%29%2E)
29. C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. 2014. Intriguing properties of neural networks. In International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=C%2E+Szegedy%2C+W%2E+Zaremba%2C+I%2E+Sutskever%2C+J%2E+Bruna%2C+D%2E+Erhan%2C+I%2E+Goodfellow%2C+and+R%2E+Fergus%2E+2014%2E+Intriguing+properties+of+neural+networks%2E+In+International+Conference+on+Learning+Representations+%28ICLR%29%2E)
30. N. Madnani and B. J. Dorr. 2010. Generating phrasal and sentential paraphrases: A survey of data-driven methods. Computational Linguistics, 36(3):341– 387. [find](https://scholar.google.com/scholar?q=N%2E+Madnani+and+B%2E+J%2E+Dorr%2E+2010%2E+Generating+phrasal+and+sentential+paraphrases%3A+A+survey+of+data%2Ddriven+methods%2E+Computational+Linguistics%2C+36%283%29%3A341%E2%80%93+387%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/madnani-2010-generating-phrasal-sentential-paraphrases-survey/)
31. S. Wang and J. Jiang. 2016. Machine comprehension using match-LSTM and answer pointer. arXiv preprint arXiv:1608.07905. [find](https://scholar.google.com/scholar?q=S%2E+Wang+and+J%2E+Jiang%2E+2016%2E+Machine+comprehension+using+match%2DLSTM+and+answer+pointer%2E+arXiv+preprint+arXiv%3A1608%2E07905%2E)
32. Z. Wang, H. Mi, W. Hamza, and R. Florian. 2016. Multi-perspective context matching for machine comprehension. arXiv. [find](https://scholar.google.com/scholar?q=Z%2E+Wang%2C+H%2E+Mi%2C+W%2E+Hamza%2C+and+R%2E+Florian%2E+2016%2E+Multi%2Dperspective+context+matching+for+machine+comprehension%2E+arXiv%2E)
33. D. Weissenborn, G. Wiese, and L. Seiffe. 2017. Making neural qa as simple as possible but not simpler. arXiv. [find](https://scholar.google.com/scholar?q=D%2E+Weissenborn%2C+G%2E+Wiese%2C+and+L%2E+Seiffe%2E+2017%2E+Making+neural+qa+as+simple+as+possible+but+not+simpler%2E+arXiv%2E)
34. Y. Yu, W. Zhang, K. Hasan, M. Yu, B. Xiang, and B. Zhou. 2016. End-to-end answer chunk extraction and ranking for reading comprehension. arXiv. [find](https://scholar.google.com/scholar?q=Y%2E+Yu%2C+W%2E+Zhang%2C+K%2E+Hasan%2C+M%2E+Yu%2C+B%2E+Xiang%2C+and+B%2E+Zhou%2E+2016%2E+End%2Dto%2Dend+answer+chunk+extraction+and+ranking+for+reading+comprehension%2E+arXiv%2E)
35. J. Zhang, X. Zhu, Q. Chen, L. Dai, S. Wei, and H. Jiang. 2017. Exploring question understanding and adaptation in neural-network-based question answering. arXiv. [find](https://scholar.google.com/scholar?q=J%2E+Zhang%2C+X%2E+Zhu%2C+Q%2E+Chen%2C+L%2E+Dai%2C+S%2E+Wei%2C+and+H%2E+Jiang%2E+2017%2E+Exploring+question+understanding+and+adaptation+in+neural%2Dnetwork%2Dbased+question+answering%2E+arXiv%2E)