[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/geiger-2020-neural-natural-language-inference-models/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / Neural Natural Language Inference Models Partially Embed Theories of Lexical Entailment and Negation

# Neural Natural Language Inference Models Partially Embed Theories of Lexical Entailment and Negation

**Authors:** Atticus Geiger, Kyle Richardson and [Christopher Potts](https://md.mova.elamur.ai/free/en/docs/papers/authors/christopher-potts/) · **Year:** 2020 · **Venue:** Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP **Link:** [aclanthology.org/2020.blackboxnlp-1.16](https://aclanthology.org/2020.blackboxnlp-1.16/) · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) (ACL Anthology)

[**44** citations](https://openalex.org/works?filter=cites:W3092665232) openalex · 2026-10-19[**68** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/geiger-2020-neural-natural-language-inference-models/) of the 140673 papers whose bibliographies we read[Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP](https://aclanthology.org/volumes/2020.blackboxnlp-1/)[licence](https://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://aclanthology.org/2020.blackboxnlp-1.16/)
- [PDF](https://aclanthology.org/2020.blackboxnlp-1.16.pdf)
- [DOI 10.18653/v1/2020.blackboxnlp-1.16](https://doi.org/10.18653/v1/2020.blackboxnlp-1.16)
- [OpenAlex](https://openalex.org/W3092665232)

**Authors (3)**

1. [Atticus Geiger](https://openalex.org/A5002577142) [ORCID](https://orcid.org/0000-0002-9170-506X) · [Semantic Scholar](https://www.semanticscholar.org/author/80833908) · h 12 · 644 citations · 79 works
2. [Kyle Richardson](https://openalex.org/A5039359682) [ORCID](https://orcid.org/0000-0003-4836-0753) · [Semantic Scholar](https://www.semanticscholar.org/author/46666605) · h 26 · 2909 citations · 127 works
3. [Christopher Potts](https://openalex.org/A5042601761) [ORCID](https://orcid.org/0000-0002-7978-6055) · [Semantic Scholar](https://www.semanticscholar.org/author/144922861) · h 54 · 28622 citations · 391 works

Citations and author metrics from openalex (2026-10-19); authors with the same name are not always told apart.

## What it is about

The question is whether neural NLI models learn how negation reverses lexical entailment: "dance" entails "move", but "not move" does not entail "not dance". The authors build MoNLI, 2,678 SNLI-style pairs in which one noun is replaced by a WordNet hypernym or hyponym. In 1,202 pairs the substitution is under "not" (NMoNLI), in 1,476 it is not (PMoNLI). They test four models: CBOW, BiLSTM, ESIM and BERT. Behaviourally, models trained on SNLI match on PMoNLI but fall well below chance on NMoNLI, and BERT is nearly always wrong there. Fine-tuning on NMoNLI, with the substituted words kept disjoint between training and test, lets all models solve the task, and BERT keeps its SNLI accuracy. Structurally, linear probes find the lexical relation at most positions, and interchange interventions at one BERT position reproduce the causal behaviour of a four-line monotonicity algorithm on subsets of 98, 63, 47 and 37 pairs. The authors conclude that BERT "at least partially" embeds the theory. The intervention search used about 7 million runs at layer 3.

## Ideas

- I1. `prag` polarity: write the monotonicity algorithm as an explicit Rust function with an enum for the relation (Forward, Reverse), so that "not" reverses the value and nothing else does. The paper's four-line algorithm is the specification; the enum follows the project's rule of constants as enums.
- I2. Generate MoNLI-style pairs from an open hypernym table (check which lexicon Mova can use), split PMoNLI and NMoNLI, and require 100% on both. The paper's below-chance NMoNLI result is the failure a rule system should not show.
- I3. Interchange test for a rule system: take two frames, swap the lexical-relation value in one, and check that the output flips exactly when negation is present. This is a causal test of Mova's own pipeline, and it is cheap.
- I4. Disjoint-word split: hold out the substituted word pairs when a lesson or rule is tuned, so that a lookup table cannot pass. Keep this split in `world/src/knowledge.rs` test items.

## For Mova

The chapters are [27. Negation](https://md.mova.elamur.ai/free/en/docs/languages/en/27-negation/) (downward monotone "not", "no", "hardly") and [19. The lexicon: vocabulary, multiword expressions and register](https://md.mova.elamur.ai/free/en/docs/languages/en/19-lexicon-mwe-register/) (hypernyms and hyponyms). The code is `prag` for polarity and entailment, `world/src/knowledge.rs` for items, and `en/src/dict.rs` for any hypernym table (grep before use, since none is confirmed here). The measure: accuracy on PMoNLI and NMoNLI separately, run twice for determinism, against a negative control in which negation is removed and the labels are swapped.

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.18653/v1/2020.blackboxnlp-1.16](https://doi.org/10.18653/v1/2020.blackboxnlp-1.16)

```
@inproceedings{geiger2020neural,
  title = {{Neural Natural Language Inference Models Partially Embed Theories of Lexical Entailment and Negation}},
  author = {Geiger, Atticus and Richardson, Kyle and Potts, Christopher},
  booktitle = {Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP},
  year = {2020},
  doi = {10.18653/v1/2020.blackboxnlp-1.16},
  url = {https://doi.org/10.18653/v1/2020.blackboxnlp-1.16}
}
```

- [← previous A Factored Functional Dependency Transformation of the English Penn Treebank for Probabilistic Surface Generation](https://md.mova.elamur.ai/free/en/docs/papers/geary-2006-factored-functional-dependency-transformation-english/)
- [next → Conversational implicatures in English dialogue: Annotated dataset](https://md.mova.elamur.ai/free/en/docs/papers/george-2019-conversational-implicatures-english-dialogue-annotated/)
- [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/)

## References

47 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Dzmitry Bahdanau, Shikhar Murty, Michael Noukhovitch, Thien Huu Nguyen, Harm de Vries, and Aaron Courville. 2018. Systematic generalization: What is required and can it be learned? In In Proceedings of the 6th International Conference on Learning Representations, Beijing. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Shikhar+Murty%2C+Michael+Noukhovitch%2C+Thien+Huu+Nguyen%2C+Harm+de+Vries%2C+and+Aaron+Courville%2E+2018%2E+Systematic+generalization%3A+What+is+required+and+can+it+be+learned%3F+In+In+Proceedings+o)
2. Yonatan Belinkov and James Glass. 2019. Analysis methods in neural language processing: A survey. Transactions of the Association for Computational Linguistics, 7:49–72. [find](https://scholar.google.com/scholar?q=Yonatan+Belinkov+and+James+Glass%2E+2019%2E+Analysis+methods+in+neural+language+processing%3A+A+survey%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+7%3A49%E2%80%9372%2E)
3. Johan van Benthem. 2008. A brief history of natural logic. In Logic, Navya-Nyaya and Applications: Homage to Bimal Matilal. [find](https://scholar.google.com/scholar?q=Johan+van+Benthem%2E+2008%2E+A+brief+history+of+natural+logic%2E+In+Logic%2C+Navya%2DNyaya+and+Applications%3A+Homage+to+Bimal+Matilal%2E)
4. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Samuel+R%2E+Bowman%2C+Gabor+Angeli%2C+Christopher+Potts%2C+and+Christopher+D%2E+Manning%2E+2015%2E+A+large+annotated+corpus+for+learning+natural+language+inference%2E+In+Proceedings+of+the+2015+Conference+on+Empirica)
5. Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, and Hui Jiang. 2016. Enhancing and combining sequential and tree LSTM for natural language inference. CoRR, abs/1609.06038. [↗](https://arxiv.org/abs/1609.06038)
6. Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019. What does BERT look at? an analysis of BERT’s attention. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 276–286, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kevin+Clark%2C+Urvashi+Khandelwal%2C+Omer+Levy%2C+and+Christopher+D%2E+Manning%2E+2019%2E+What+does+BERT+look+at%3F+an+analysis+of+BERT%E2%80%99s+attention%2E+In+Proceedings+of+the+2019+ACL+Workshop+BlackboxNLP%3A+Analyzing+an)
7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
8. Christiane Fellbaum, editor. 1998. WordNet: An Electronic Database. MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=Christiane+Fellbaum%2C+editor%2E+1998%2E+WordNet%3A+An+Electronic+Database%2E+MIT+Press%2C+Cambridge%2C+MA%2E)
9. Jerry A. Fodor and Zenon W. Pylyshyn. 1988. Connectionism and cognitive architecture: A critical analysis. Cognition, 28(1):3–71. [find](https://scholar.google.com/scholar?q=Jerry+A%2E+Fodor+and+Zenon+W%2E+Pylyshyn%2E+1988%2E+Connectionism+and+cognitive+architecture%3A+A+critical+analysis%2E+Cognition%2C+28%281%29%3A3%E2%80%9371%2E)
10. Atticus Geiger, Ignacio Cases, Lauri Karttunen, and Christopher Potts. 2019. Posing fair generalization tasks for natural language inference. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4485–4495, Hong Kong, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Atticus+Geiger%2C+Ignacio+Cases%2C+Lauri+Karttunen%2C+and+Christopher+Potts%2E+2019%2E+Posing+fair+generalization+tasks+for+natural+language+inference%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods)
11. Mario Giulianelli, Jack Harding, Florian Mohnert, Dieuwke Hupkes, and Willem Zuidema. 2018. Under the hood: Using diagnostic classifiers to investigate and improve how language models track agreement information. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 240–248, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mario+Giulianelli%2C+Jack+Harding%2C+Florian+Mohnert%2C+Dieuwke+Hupkes%2C+and+Willem+Zuidema%2E+2018%2E+Under+the+hood%3A+Using+diagnostic+classifiers+to+investigate+and+improve+how+language+models+track+agreement+)
12. Max Glockner, Vered Shwartz, and Yoav Goldberg. 2018. Breaking NLI systems with sentences that require simple lexical inferences. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 650–655, Melbourne, Australia. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Max+Glockner%2C+Vered+Shwartz%2C+and+Yoav+Goldberg%2E+2018%2E+Breaking+NLI+systems+with+sentences+that+require+simple+lexical+inferences%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Compu) [our card](https://md.mova.elamur.ai/free/en/docs/papers/glockner-2018-breaking-nli-systems-sentences-that/)
13. Emily Goodwin, Koustuv Sinha, and Timothy J. O’Donnell. 2020. Probing linguistic systematicity. [find](https://scholar.google.com/scholar?q=Emily+Goodwin%2C+Koustuv+Sinha%2C+and+Timothy+J%2E+O%E2%80%99Donnell%2E+2020%2E+Probing+linguistic+systematicity%2E)
14. Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018. Annotation artifacts in natural language inference data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 107–112, New Orleans, Louisiana. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Suchin+Gururangan%2C+Swabha+Swayamdipta%2C+Omer+Levy%2C+Roy+Schwartz%2C+Samuel+Bowman%2C+and+Noah+A%2E+Smith%2E+2018%2E+Annotation+artifacts+in+natural+language+inference+data%2E+In+Proceedings+of+the+2018+Conference+o)
15. John Hewitt and Percy Liang. 2019. Designing and interpreting probes with control tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2733–2743, Hong Kong, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=John+Hewitt+and+Percy+Liang%2E+2019%2E+Designing+and+interpreting+probes+with+control+tasks%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+the+9th+Internati)
16. Hai Hu, Qi Chen, and Larry Moss. 2019a. Natural language inference with monotonicity. In Proceedings of the 13th International Conference on Computational Semantics - Short Papers, pages 8– 15, Gothenburg, Sweden. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hai+Hu%2C+Qi+Chen%2C+and+Larry+Moss%2E+2019a%2E+Natural+language+inference+with+monotonicity%2E+In+Proceedings+of+the+13th+International+Conference+on+Computational+Semantics+%2D+Short+Papers%2C+pages+8%E2%80%93+15%2C+Gothen)
17. Hai Hu, Qi Chen, Kyle Richardson, Atreyee Mukherjee, Lawrence S. Moss, and Sandra Kübler. 2019b. MonaLog: A lightweight system for natural language inference based on monotonicity. ArXiv, abs/1910.08772. [↗](https://arxiv.org/abs/1910.08772)
18. Dieuwke Hupkes, Sanne Bouwmeester, and Raquel Fernández. 2018. Analysing the potential of seq-to-seq models for incremental interpretation in task-oriented dialogue. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 165–174, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Dieuwke+Hupkes%2C+Sanne+Bouwmeester%2C+and+Raquel+Ferna%CC%81ndez%2E+2018%2E+Analysing+the+potential+of+seq%2Dto%2Dseq+models+for+incremental+interpretation+in+task%2Doriented+dialogue%2E+In+Proceedings+of+the+2018+EMNLP+)
19. Dieuwke Hupkes, Verna Dankers, Mathijs Mul, and Elia Bruni. 2019. Compositionality decomposed: how do neural networks generalise? [find](https://scholar.google.com/scholar?q=Dieuwke+Hupkes%2C+Verna+Dankers%2C+Mathijs+Mul%2C+and+Elia+Bruni%2E+2019%2E+Compositionality+decomposed%3A+how+do+neural+networks+generalise%3F)
20. Thomas Icard, Lawrence Moss, and William Tune. 2017. A monotonicity calculus and its completeness. In Proceedings of the 15th Meeting on the Mathematics of Language, pages 75–87, London, UK. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Thomas+Icard%2C+Lawrence+Moss%2C+and+William+Tune%2E+2017%2E+A+monotonicity+calculus+and+its+completeness%2E+In+Proceedings+of+the+15th+Meeting+on+the+Mathematics+of+Language%2C+pages+75%E2%80%9387%2C+London%2C+UK%2E+Associati)
21. Thomas F. Icard. 2012. Inclusion and exclusion in natural language. Studia Logica, 100(4):705–725. [find](https://scholar.google.com/scholar?q=Thomas+F%2E+Icard%2E+2012%2E+Inclusion+and+exclusion+in+natural+language%2E+Studia+Logica%2C+100%284%29%3A705%E2%80%93725%2E)
22. Thomas F. Icard. 2017. From programs to causal models. In Proceedings of the 21st Amsterdam Colloquium, pages 35–44. University of Amsterdam. [find](https://scholar.google.com/scholar?q=Thomas+F%2E+Icard%2E+2017%2E+From+programs+to+causal+models%2E+In+Proceedings+of+the+21st+Amsterdam+Colloquium%2C+pages+35%E2%80%9344%2E+University+of+Amsterdam%2E)
23. Thomas F. Icard and Lawrence S. Moss. 2013. Recent progress on monotonicity. Linguistic Issues in Language Technology, 9(7):1–31. [find](https://scholar.google.com/scholar?q=Thomas+F%2E+Icard+and+Lawrence+S%2E+Moss%2E+2013%2E+Recent+progress+on+monotonicity%2E+Linguistic+Issues+in+Language+Technology%2C+9%287%29%3A1%E2%80%9331%2E)
24. Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. CoRR, abs/1707.07328. [↗](https://arxiv.org/abs/1707.07328) [our card](https://md.mova.elamur.ai/free/en/docs/papers/jia-2017-adversarial-examples-evaluating-reading-comprehension/)
25. Brenden M. Lake and Marco Baroni. 2018. Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2879–2888. PMLR. [find](https://scholar.google.com/scholar?q=Brenden+M%2E+Lake+and+Marco+Baroni%2E+2018%2E+Generalization+without+systematicity%3A+On+the+compositional+skills+of+sequence%2Dto%2Dsequence+recurrent+networks%2E+In+Proceedings+of+the+35th+International+Conferenc)
26. Nelson F. Liu, Roy Schwartz, and Noah A. Smith. 2019. Inoculation by fine-tuning: A method for analyzing challenge datasets. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2171–2179, Minneapolis, Minnesota. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Nelson+F%2E+Liu%2C+Roy+Schwartz%2C+and+Noah+A%2E+Smith%2E+2019%2E+Inoculation+by+fine%2Dtuning%3A+A+method+for+analyzing+challenge+datasets%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter+of+the+)
27. Bill MacCartney. 2009. Natural Language Inference. Ph.D. thesis, Stanford University. [find](https://scholar.google.com/scholar?q=Bill+MacCartney%2E+2009%2E+Natural+Language+Inference%2E+Ph%2ED%2E+thesis%2C+Stanford+University%2E)
28. Bill MacCartney and Christopher D. Manning. 2008. Modeling semantic containment and exclusion in natural language inference. In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 521–528, Manchester, UK. Coling 2008 Organizing Committee. [find](https://scholar.google.com/scholar?q=Bill+MacCartney+and+Christopher+D%2E+Manning%2E+2008%2E+Modeling+semantic+containment+and+exclusion+in+natural+language+inference%2E+In+Proceedings+of+the+22nd+International+Conference+on+Computational+Lingui)
29. Bill MacCartney and Christopher D. Manning. 2009. An extended model of natural logic. In Proceedings of the Eight International Conference on Computational Semantics, pages 140–156, Tilburg, The Netherlands. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Bill+MacCartney+and+Christopher+D%2E+Manning%2E+2009%2E+An+extended+model+of+natural+logic%2E+In+Proceedings+of+the+Eight+International+Conference+on+Computational+Semantics%2C+pages+140%E2%80%93156%2C+Tilburg%2C+The+Nethe)
30. Aakanksha Naik, Abhilasha Ravichander, Norman Sadeh, Carolyn Rose, and Graham Neubig. 2018. Stress test evaluation for natural language inference. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2340–2353, Santa Fe, New Mexico, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Aakanksha+Naik%2C+Abhilasha+Ravichander%2C+Norman+Sadeh%2C+Carolyn+Rose%2C+and+Graham+Neubig%2E+2018%2E+Stress+test+evaluation+for+natural+language+inference%2E+In+Proceedings+of+the+27th+International+Conference+o)
31. Yixin Nie, Yicheng Wang, and Mohit Bansal. 2019a. Analyzing compositionality-sensitivity of NLI models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6867–6874. [find](https://scholar.google.com/scholar?q=Yixin+Nie%2C+Yicheng+Wang%2C+and+Mohit+Bansal%2E+2019a%2E+Analyzing+compositionality%2Dsensitivity+of+NLI+models%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+33%2C+pages+6867%E2%80%936874%2E)
32. Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2019b. Adversarial NLI: A new benchmark for natural language understanding. [find](https://scholar.google.com/scholar?q=Yixin+Nie%2C+Adina+Williams%2C+Emily+Dinan%2C+Mohit+Bansal%2C+Jason+Weston%2C+and+Douwe+Kiela%2E+2019b%2E+Adversarial+NLI%3A+A+new+benchmark+for+natural+language+understanding%2E)
33. Ellie Pavlick and Chris Callison-Burch. 2016. Most “babies” are “little” and most “problems” are “huge”: Compositional entailment in adjective-nouns. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2164–2173, Berlin, Germany. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ellie+Pavlick+and+Chris+Callison%2DBurch%2E+2016%2E+Most+%E2%80%9Cbabies%E2%80%9D+are+%E2%80%9Clittle%E2%80%9D+and+most+%E2%80%9Cproblems%E2%80%9D+are+%E2%80%9Chuge%E2%80%9D%3A+Compositional+entailment+in+adjective%2Dnouns%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+A)
34. Judea Pearl. 2001. Direct and indirect effects. In Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, UAI’01, page 411–420, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc. [find](https://scholar.google.com/scholar?q=Judea+Pearl%2E+2001%2E+Direct+and+indirect+effects%2E+In+Proceedings+of+the+Seventeenth+Conference+on+Uncertainty+in+Artificial+Intelligence%2C+UAI%E2%80%9901%2C+page+411%E2%80%93420%2C+San+Francisco%2C+CA%2C+USA%2E+Morgan+Kaufmann+Pu)
35. Matthew Peters, Mark Neumann, Luke Zettlemoyer, and Wen-tau Yih. 2018. Dissecting contextual word embeddings: Architecture and representation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1499–1509, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Matthew+Peters%2C+Mark+Neumann%2C+Luke+Zettlemoyer%2C+and+Wen%2Dtau+Yih%2E+2018%2E+Dissecting+contextual+word+embeddings%3A+Architecture+and+representation%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Method)
36. Kyle Richardson, Hai Hu, Lawrence S. Moss, and Ashish Sabharwal. 2019. Probing natural language inference models through semantic fragments. [find](https://scholar.google.com/scholar?q=Kyle+Richardson%2C+Hai+Hu%2C+Lawrence+S%2E+Moss%2C+and+Ashish+Sabharwal%2E+2019%2E+Probing+natural+language+inference+models+through+semantic+fragments%2E)
37. Vı́ctor Sánchez-Valencia. 1991. Studies in Natural Logic and Categorial Grammar. Ph.D. thesis, University of Amsterdam. [find](https://scholar.google.com/scholar?q=V%C4%B1%CC%81ctor+Sa%CC%81nchez%2DValencia%2E+1991%2E+Studies+in+Natural+Logic+and+Categorial+Grammar%2E+Ph%2ED%2E+thesis%2C+University+of+Amsterdam%2E)
38. Alon Talmor, Yanai Elazar, Yoav Goldberg, and Jonathan Berant. 2019. olmpics – on what language model pre-training captures. [find](https://scholar.google.com/scholar?q=Alon+Talmor%2C+Yanai+Elazar%2C+Yoav+Goldberg%2C+and+Jonathan+Berant%2E+2019%2E+olmpics+%E2%80%93+on+what+language+model+pre%2Dtraining+captures%2E)
39. David Marr. 1982. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. Henry Holt and Co., Inc., New York, NY, USA. [find](https://scholar.google.com/scholar?q=David+Marr%2E+1982%2E+Vision%3A+A+Computational+Investigation+into+the+Human+Representation+and+Processing+of+Visual+Information%2E+Henry+Holt+and+Co%2E%2C+Inc%2E%2C+New+York%2C+NY%2C+USA%2E)
40. Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. BERT rediscovers the classical NLP pipeline. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4593– 4601, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ian+Tenney%2C+Dipanjan+Das%2C+and+Ellie+Pavlick%2E+2019%2E+BERT+rediscovers+the+classical+NLP+pipeline%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+4593%E2%80%93+)
41. Lawrence S Moss. 2009. Natural logic and semantics. In Proceedings of the 18th Amsterdam Colloquium: Revised Selected Papers, pages 71–80, Berlin. University of Amsterdam, Springer. [find](https://scholar.google.com/scholar?q=Lawrence+S+Moss%2E+2009%2E+Natural+logic+and+semantics%2E+In+Proceedings+of+the+18th+Amsterdam+Colloquium%3A+Revised+Selected+Papers%2C+pages+71%E2%80%9380%2C+Berlin%2E+University+of+Amsterdam%2C+Springer%2E)
42. Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020. Causal mediation analysis for interpreting neural nlp: The case of gender bias. [find](https://scholar.google.com/scholar?q=Jesse+Vig%2C+Sebastian+Gehrmann%2C+Yonatan+Belinkov%2C+Sharon+Qian%2C+Daniel+Nevo%2C+Yaron+Singer%2C+and+Stuart+Shieber%2E+2020%2E+Causal+mediation+analysis+for+interpreting+neural+nlp%3A+The+case+of+gender+bias%2E)
43. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112–1122. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Nikita+Nangia%2C+and+Samuel+Bowman%2E+2018%2E+A+broad%2Dcoverage+challenge+corpus+for+sentence+understanding+through+inference%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chap)
44. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. Huggingface’s transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771. [↗](https://arxiv.org/abs/1910.03771)
45. Hitomi Yanaka, Koji Mineshima, Daisuke Bekki, and Kentaro Inui. 2020. Do neural models learn systematicity of monotonicity inference in natural language? [find](https://scholar.google.com/scholar?q=Hitomi+Yanaka%2C+Koji+Mineshima%2C+Daisuke+Bekki%2C+and+Kentaro+Inui%2E+2020%2E+Do+neural+models+learn+systematicity+of+monotonicity+inference+in+natural+language%3F)
46. Hitomi Yanaka, Koji Mineshima, Daisuke Bekki, Kentaro Inui, Satoshi Sekine, Lasha Abzianidze, and Johan Bos. 2019a. Can neural networks understand monotonicity reasoning? In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 31–40, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hitomi+Yanaka%2C+Koji+Mineshima%2C+Daisuke+Bekki%2C+Kentaro+Inui%2C+Satoshi+Sekine%2C+Lasha+Abzianidze%2C+and+Johan+Bos%2E+2019a%2E+Can+neural+networks+understand+monotonicity+reasoning%3F+In+Proceedings+of+the+2019+AC)
47. Hitomi Yanaka, Koji Mineshima, Daisuke Bekki, Kentaro Inui, Satoshi Sekine, Lasha Abzianidze, and Johan Bos. 2019b. HELP: A dataset for identifying shortcomings of neural models in monotonicity reasoning. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), pages 250–255, Minneapolis, Minnesota. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hitomi+Yanaka%2C+Koji+Mineshima%2C+Daisuke+Bekki%2C+Kentaro+Inui%2C+Satoshi+Sekine%2C+Lasha+Abzianidze%2C+and+Johan+Bos%2E+2019b%2E+HELP%3A+A+dataset+for+identifying+shortcomings+of+neural+models+in+monotonicity+reason)