[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/vashishtha-2020-temporal-reasoning-natural-language-inference/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / Temporal Reasoning in Natural Language Inference

# Temporal Reasoning in Natural Language Inference

**Authors:** Siddharth Vashishtha, Adam Poliak, [Yash Kumar Lal](https://md.mova.elamur.ai/free/en/docs/papers/authors/yash-kumar-lal/), [Benjamin Van Durme](https://md.mova.elamur.ai/free/en/docs/papers/authors/benjamin-van-durme/), [Aaron Steven White](https://md.mova.elamur.ai/free/en/docs/papers/authors/aaron-steven-white/) · **Year:** 2020 · **Venue:** Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 4070–4078 **Link:** [aclanthology.org/2020.findings-emnlp.363](https://aclanthology.org/2020.findings-emnlp.363/) (DOI [10.18653/v1/2020.findings-emnlp.363](https://doi.org/10.18653/v1/2020.findings-emnlp.363)) · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) (ACL Anthology)

[**31** citations](https://openalex.org/works?filter=cites:W3104036557) openalex · 2026-10-18[**28** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/vashishtha-2020-temporal-reasoning-natural-language-inference/) of the 140673 papers whose bibliographies we read[Findings of the Association for Computational Linguistics: EMNLP 2020](https://aclanthology.org/volumes/2020.findings-emnlp/)[licence](https://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://aclanthology.org/2020.findings-emnlp.363/)
- [PDF](https://aclanthology.org/2020.findings-emnlp.363.pdf)
- [DOI 10.18653/v1/2020.findings-emnlp.363](https://doi.org/10.18653/v1/2020.findings-emnlp.363)
- [OpenAlex](https://openalex.org/W3104036557)

**Authors (5)**

1. [Siddharth Vashishtha](https://openalex.org/A5015768886) [ORCID](https://orcid.org/0000-0001-9025-6584) · h 6 · 209 citations · 18 works
2. [Adam Poliak](https://openalex.org/A5032380221) [ORCID](https://orcid.org/0000-0002-1903-8450) · h 21 · 4508 citations · 51 works
3. [Yash Kumar Lal](https://openalex.org/A5064437096) h 8 · 254 citations · 43 works
4. [Benjamin Van Durme](https://openalex.org/A5075825791) [ORCID](https://orcid.org/0000-0003-4328-4288) · h 51 · 9972 citations · 392 works
5. [Aaron Steven White](https://openalex.org/A5090639239) [ORCID](https://orcid.org/0000-0003-0057-9246) · h 19 · 1185 citations · 69 works

Citations and author metrics from openalex (2026-10-18); authors with the same name are not always told apart.

## What it is about

The paper turns four temporal datasets (TempEval3, TimeBank-Dense, RED and UDS-Time) into NLI pairs about two kinds of temporal reasoning: event order and event duration. Order is asked through 8 templates about the start and end points of two events, mapped from Allen-style relations. Duration is asked through 18 templates ("X did last longer than a minute"), built from the 11 duration labels of UDS-Time. The recast sets have more than a million pairs in total (504,136 duration pairs and 562,944 order pairs from UDS-Time, and 27,240, 11,910 and 5,578 from the other three). A validation study with 48 crowd workers finds that entailed pairs are rated as more likely true (χ²(1) = 226.1, p < 0.001). Models trained on MNLI do poorly on these sets, often below the majority baseline. Duration is partly solvable by templates alone: a template-based majority reaches 80.2% on the duration test set, and the neural models gain about 10 points over it.

## Ideas

- I1. Temporal rules: the 8 order templates become deterministic rules over start and end points in `world/src/events.rs`, so that "X ended before Y started" is answered from the ordered world, with the rule that fired. Measure exact match on school stories with hand-labelled order.
- I2. Duration bands: store a typical duration band for each verb class (from seconds to centuries, the paper's 11 labels) as a BTreeMap in `world/`. The paper's template baseline shows how much the label skew alone gives, so report the skew before any learning.
- I3. Validation as a count test: the paper uses an ordinal mixed-effects model. For Mova, compare mean ratings of entailed and not-entailed pairs with a permutation test, and use shuffled labels as the negative control.
- I4. The MNLI results show that generic NLI training does not teach temporal order. Temporal rules must therefore be explicit in Mova, and the chapter that should hold them is [36. Discourse relations and coherence](https://md.mova.elamur.ai/free/en/docs/languages/en/36-discourse-relations/).

## For Mova

The code is `world/src/events.rs` and `world/src/narr.rs`. The chapters are [36. Discourse relations and coherence](https://md.mova.elamur.ai/free/en/docs/languages/en/36-discourse-relations/) and [13. Verbs](https://md.mova.elamur.ai/free/en/docs/languages/en/13-verbs/) for tense and aspect. Measure order and duration accuracy on school stories, with the two-fold `cv` split, run twice.

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.18653/v1/2020.findings-emnlp.363](https://doi.org/10.18653/v1/2020.findings-emnlp.363)

```
@inproceedings{vashishtha2020temporal,
  title = {{Temporal Reasoning in Natural Language Inference}},
  author = {Vashishtha, Siddharth and Poliak, Adam and Lal, Yash Kumar and Durme, Benjamin Van and White, Aaron Steven},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2020},
  year = {2020},
  doi = {10.18653/v1/2020.findings-emnlp.363},
  url = {https://doi.org/10.18653/v1/2020.findings-emnlp.363}
}
```

- [← previous ARRAU: Linguistically-Motivated Annotation of Anaphoric Descriptions](https://md.mova.elamur.ai/free/en/docs/papers/uryupina-2016-arrau-linguistically-motivated-annotation-anaphoric/)
- [next → Lemmatisation, etymology and information overload on English and Swedish editions of Wiktionary](https://md.mova.elamur.ai/free/en/docs/papers/verdizade-2021-wiktionary-en-sv/)
- [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/)

## References

59 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. James F Allen. 1983. Maintaining knowledge about temporal intervals. Communications of the ACM, 26(11):832–843. [find](https://scholar.google.com/scholar?q=James+F+Allen%2E+1983%2E+Maintaining+knowledge+about+temporal+intervals%2E+Communications+of+the+ACM%2C+26%2811%29%3A832%E2%80%93843%2E)
2. James F Allen. 1984. Towards a general theory of action and time. Artificial intelligence, 23(2):123– 154. [find](https://scholar.google.com/scholar?q=James+F+Allen%2E+1984%2E+Towards+a+general+theory+of+action+and+time%2E+Artificial+intelligence%2C+23%282%29%3A123%E2%80%93+154%2E)
3. Yonatan Belinkov, Adam Poliak, Stuart M Shieber, Benjamin Van Durme, and Alexander M Rush. 2019. Don’t take the premise for granted: Mitigating artifacts in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 877–891. [find](https://scholar.google.com/scholar?q=Yonatan+Belinkov%2C+Adam+Poliak%2C+Stuart+M+Shieber%2C+Benjamin+Van+Durme%2C+and+Alexander+M+Rush%2E+2019%2E+Don%E2%80%99t+take+the+premise+for+granted%3A+Mitigating+artifacts+in+natural+language+inference%2E+In+Proceedings+)
4. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Samuel+R%2E+Bowman%2C+Gabor+Angeli%2C+Christopher+Potts%2C+and+Christopher+D%2E+Manning%2E+2015%2E+A+large+annotated+corpus+for+learning+natural+language+inference%2E+In+Proceedings+of+the+2015+Conference+on+Empirica)
5. Bertram C Bruce. 1972. A model for temporal references and its appliicationin a question answering program. Artificial intelligence. [find](https://scholar.google.com/scholar?q=Bertram+C+Bruce%2E+1972%2E+A+model+for+temporal+references+and+its+appliicationin+a+question+answering+program%2E+Artificial+intelligence%2E)
6. Roberto Casati and Achille C. Varzi. 1996. Events. Dartmouth. [find](https://scholar.google.com/scholar?q=Roberto+Casati+and+Achille+C%2E+Varzi%2E+1996%2E+Events%2E+Dartmouth%2E)
7. Nathanael Chambers, Taylor Cassidy, Bill McDowell, and Steven Bethard. 2014. Dense event ordering with a multi-pass architecture. Transactions of the Association for Computational Linguistics, 2:273– 284. [find](https://scholar.google.com/scholar?q=Nathanael+Chambers%2C+Taylor+Cassidy%2C+Bill+McDowell%2C+and+Steven+Bethard%2E+2014%2E+Dense+event+ordering+with+a+multi%2Dpass+architecture%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+2%3A273%E2%80%93+)
8. Yao Cheng, Peter Anick, Pengyu Hong, and Nianwen Xue. 2013. Temporal relation discovery between events and temporal expressions identified in clinical narrative. Journal of biomedical informatics, 46:S48–S53. [find](https://scholar.google.com/scholar?q=Yao+Cheng%2C+Peter+Anick%2C+Pengyu+Hong%2C+and+Nianwen+Xue%2E+2013%2E+Temporal+relation+discovery+between+events+and+temporal+expressions+identified+in+clinical+narrative%2E+Journal+of+biomedical+informatics%2C+46%3A)
9. Alexis Conneau, Douwe Kiela, Holger Schwenk, Loïc Barrault, and Antoine Bordes. 2017. Supervised learning of universal sentence representations from natural language inference data. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 670–680, Copenhagen, Denmark. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Douwe+Kiela%2C+Holger+Schwenk%2C+Lo%C3%AFc+Barrault%2C+and+Antoine+Bordes%2E+2017%2E+Supervised+learning+of+universal+sentence+representations+from+natural+language+inference+data%2E+In+Proceedings+of+)
10. Robin Cooper, Dick Crouch, Jan Van Eijck, Chris Fox, Johan Van Genabith, Jan Jaspars, Hans Kamp, David Milward, Manfred Pinkal, Massimo Poesio, et al. 1996. Using the framework. Technical report. [find](https://scholar.google.com/scholar?q=Robin+Cooper%2C+Dick+Crouch%2C+Jan+Van+Eijck%2C+Chris+Fox%2C+Johan+Van+Genabith%2C+Jan+Jaspars%2C+Hans+Kamp%2C+David+Milward%2C+Manfred+Pinkal%2C+Massimo+Poesio%2C+et+al%2E+1996%2E+Using+the+framework%2E+Technical+report%2E)
11. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2006. The pascal recognising textual entailment challenge. In Machine learning challenges. evaluating predictive uncertainty, visual object classification, and recognising tectual entailment, pages 177– 190. Springer. [find](https://scholar.google.com/scholar?q=Ido+Dagan%2C+Oren+Glickman%2C+and+Bernardo+Magnini%2E+2006%2E+The+pascal+recognising+textual+entailment+challenge%2E+In+Machine+learning+challenges%2E+evaluating+predictive+uncertainty%2C+visual+object+classificati)
12. David Graff. The aquaint corpus of English news text:[content copyright] Portions c 1998-2000 New York Times, Inc., c 1998-2000 Associated Press, Inc., c 1996-2000 Xinhua News Service. Linguistic Data Consortium. [find](https://scholar.google.com/scholar?q=David+Graff%2E+The+aquaint+corpus+of+English+news+text%3A%5Bcontent+copyright%5D+Portions+c+1998%2D2000+New+York+Times%2C+Inc%2E%2C+c+1998%2D2000+Associated+Press%2C+Inc%2E%2C+c+1996%2D2000+Xinhua+News+Service%2E+Linguistic+Data)
13. Rujun Han, I-Hung Hsu, Mu Yang, Aram Galstyan, Ralph Weischedel, and Nanyun Peng. 2019. Deep structured neural network for event temporal relation extraction. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 666–106, Hong Kong, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Rujun+Han%2C+I%2DHung+Hsu%2C+Mu+Yang%2C+Aram+Galstyan%2C+Ralph+Weischedel%2C+and+Nanyun+Peng%2E+2019%2E+Deep+structured+neural+network+for+event+temporal+relation+extraction%2E+In+Proceedings+of+the+23rd+Conference+on+)
14. Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III. 2015. Deep unordered composition rivals syntactic methods for text classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1681–1691. [find](https://scholar.google.com/scholar?q=Mohit+Iyyer%2C+Varun+Manjunatha%2C+Jordan+Boyd%2DGraber%2C+and+Hal+Daum%C3%A9+III%2E+2015%2E+Deep+unordered+composition+rivals+syntactic+methods+for+text+classification%2E+In+Proceedings+of+the+53rd+Annual+Meeting+of+th)
15. Paloma Jeretic, Alex Warstadt, Suvrat Bhooshan, and Adina Williams. 2020. Are natural language inference models imppressive? learning implicature and presupposition. arXiv preprint arXiv:2004.03066. [↗](https://arxiv.org/abs/2004.03066) [our card](https://md.mova.elamur.ai/free/en/docs/papers/jeretic-2020-are-natural-language-inference-models/)
16. Hyuckchul Jung, James Allen, Nate Blaylock, William de Beaumont, Lucian Galescu, and Mary Swift. 2011. Building timelines from narrative clinical records: initial results based-on deep natural language understanding. In Proceedings of BioNLP 2011 workshop, pages 146–154. [find](https://scholar.google.com/scholar?q=Hyuckchul+Jung%2C+James+Allen%2C+Nate+Blaylock%2C+William+de+Beaumont%2C+Lucian+Galescu%2C+and+Mary+Swift%2E+2011%2E+Building+timelines+from+narrative+clinical+records%3A+initial+results+based%2Don+deep+natural+languag)
17. Kenneth Kahn and G Anthony Gorry. 1977. Mechanizing temporal knowledge. Artificial intelligence, 9(1):87–108. [find](https://scholar.google.com/scholar?q=Kenneth+Kahn+and+G+Anthony+Gorry%2E+1977%2E+Mechanizing+temporal+knowledge%2E+Artificial+intelligence%2C+9%281%29%3A87%E2%80%93108%2E)
18. Daniel Khashabi. 2019. Reasoning-Driven Question-Answering for Natural Language Understanding. Ph.D. thesis, University of Pennsylvania. [find](https://scholar.google.com/scholar?q=Daniel+Khashabi%2E+2019%2E+Reasoning%2DDriven+Question%2DAnswering+for+Natural+Language+Understanding%2E+Ph%2ED%2E+thesis%2C+University+of+Pennsylvania%2E)
19. Christo Kirov, Ryan Cotterell, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Patrick Xia, Manaal Faruqui, Sabrina J Mielke, Arya D McCarthy, Sandra Kübler, et al. 2018. Unimorph 2.0: universal morphology. arXiv preprint arXiv:1810.11101. [↗](https://arxiv.org/abs/1810.11101)
20. Robert Kowalski and Marek Sergot. 1989. A logic-based calculus of events. In Foundations of knowledge base management, pages 23–55. Springer. [find](https://scholar.google.com/scholar?q=Robert+Kowalski+and+Marek+Sergot%2E+1989%2E+A+logic%2Dbased+calculus+of+events%2E+In+Foundations+of+knowledge+base+management%2C+pages+23%E2%80%9355%2E+Springer%2E)
21. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. [↗](https://arxiv.org/abs/1907.11692)
22. Hector Llorens, Nathanael Chambers, Naushad Uz-Zaman, Nasrin Mostafazadeh, James Allen, and James Pustejovsky. 2015. Semeval-2015 task 5: Qa tempeval-evaluating temporal information understanding with question answering. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 792–800. [find](https://scholar.google.com/scholar?q=Hector+Llorens%2C+Nathanael+Chambers%2C+Naushad+Uz%2DZaman%2C+Nasrin+Mostafazadeh%2C+James+Allen%2C+and+James+Pustejovsky%2E+2015%2E+Semeval%2D2015+task+5%3A+Qa+tempeval%2Devaluating+temporal+information+understanding+with)
23. John McCarthy and Patrick J Hayes. 1987. Some philosophical problems from the standpoint of artificial intelligence. In Readings in nonmonotonic reasoning, pages 26–45. [find](https://scholar.google.com/scholar?q=John+McCarthy+and+Patrick+J+Hayes%2E+1987%2E+Some+philosophical+problems+from+the+standpoint+of+artificial+intelligence%2E+In+Readings+in+nonmonotonic+reasoning%2C+pages+26%E2%80%9345%2E)
24. Drew McDermott. 1982. A temporal logic for reasoning about processes and plans. Cognitive science, 6(2):101–155. [find](https://scholar.google.com/scholar?q=Drew+McDermott%2E+1982%2E+A+temporal+logic+for+reasoning+about+processes+and+plans%2E+Cognitive+science%2C+6%282%29%3A101%E2%80%93155%2E)
25. Yuanliang Meng and Anna Rumshisky. 2018. Context-aware neural model for temporal information extraction. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 527–536, Melbourne, Australia. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yuanliang+Meng+and+Anna+Rumshisky%2E+2018%2E+Context%2Daware+neural+model+for+temporal+information+extraction%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Vol)
26. Nasrin Mostafazadeh, Alyson Grealish, Nathanael Chambers, James Allen, and Lucy Vanderwende. 2016. CaTeRS: Causal and temporal relation scheme for semantic annotation of event structures. In Proceedings of the Fourth Workshop on Events, pages 51–61, San Diego, California. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Nasrin+Mostafazadeh%2C+Alyson+Grealish%2C+Nathanael+Chambers%2C+James+Allen%2C+and+Lucy+Vanderwende%2E+2016%2E+CaTeRS%3A+Causal+and+temporal+relation+scheme+for+semantic+annotation+of+event+structures%2E+In+Proceedin)
27. Lili Mou, Rui Men, Ge Li, Yan Xu, Lu Zhang, Rui Yan, and Zhi Jin. 2016. Natural language inference by tree-based convolution and heuristic matching. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 130–136, Berlin, Germany. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Lili+Mou%2C+Rui+Men%2C+Ge+Li%2C+Yan+Xu%2C+Lu+Zhang%2C+Rui+Yan%2C+and+Zhi+Jin%2E+2016%2E+Natural+language+inference+by+tree%2Dbased+convolution+and+heuristic+matching%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+As)
28. Aakanksha Naik, Luke Breitfeller, and Carolyn Rose. 2019. TDDiscourse: A dataset for discourse-level temporal ordering of events. In Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue, pages 239–249, Stockholm, Sweden. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Aakanksha+Naik%2C+Luke+Breitfeller%2C+and+Carolyn+Rose%2E+2019%2E+TDDiscourse%3A+A+dataset+for+discourse%2Dlevel+temporal+ordering+of+events%2E+In+Proceedings+of+the+20th+Annual+SIGdial+Meeting+on+Discourse+and+Dia)
29. Alexander Nakhimovsky. 1987. Temporal reasoning in natural language understanding: The temporal structure of the narrative. In Proceedings of the Third Conference on European Chapter of the Association for Computational Linguistics, EACL ’87, page 262–269, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Alexander+Nakhimovsky%2E+1987%2E+Temporal+reasoning+in+natural+language+understanding%3A+The+temporal+structure+of+the+narrative%2E+In+Proceedings+of+the+Third+Conference+on+European+Chapter+of+the+Associatio)
30. Qiang Ning, Zhili Feng, and Dan Roth. 2017. A structured learning approach to temporal relation extraction. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1027–1037. [find](https://scholar.google.com/scholar?q=Qiang+Ning%2C+Zhili+Feng%2C+and+Dan+Roth%2E+2017%2E+A+structured+learning+approach+to+temporal+relation+extraction%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+p)
31. Qiang Ning, Zhili Feng, Hao Wu, and Dan Roth. 2018a. Joint reasoning for temporal and causal relations. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 2278–2288. [find](https://scholar.google.com/scholar?q=Qiang+Ning%2C+Zhili+Feng%2C+Hao+Wu%2C+and+Dan+Roth%2E+2018a%2E+Joint+reasoning+for+temporal+and+causal+relations%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volu)
32. Qiang Ning, Hao Wu, Rujun Han, Nanyun Peng, Matt Gardner, and Dan Roth. 2020. Torque: A reading comprehension dataset of temporal ordering questions. arXiv preprint arXiv:2005.00242. [↗](https://arxiv.org/abs/2005.00242) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ning-2020-torque-reading-comprehension-dataset-temporal/)
33. Qiang Ning, Hao Wu, and Dan Roth. 2018b. A multi-axis annotation scheme for event temporal relations. arXiv preprint arXiv:1804.07828. [↗](https://arxiv.org/abs/1804.07828)
34. Tim O’Gorman, Kristin Wright-Bettner, and Martha Palmer. 2016. Richer event description: Integrating event coreference with temporal, causal and bridging annotation. In Proceedings of the 2nd Workshop on Computing News Storylines (CNS 2016), pages 47–56. [find](https://scholar.google.com/scholar?q=Tim+O%E2%80%99Gorman%2C+Kristin+Wright%2DBettner%2C+and+Martha+Palmer%2E+2016%2E+Richer+event+description%3A+Integrating+event+coreference+with+temporal%2C+causal+and+bridging+annotation%2E+In+Proceedings+of+the+2nd+Workshop)
35. AK Pani and GP Bhattacharjee. 2001. Temporal representation and reasoning in artificial intelligence: A review. Mathematical and Computer Modelling, 34(1-2):55–80. [find](https://scholar.google.com/scholar?q=AK+Pani+and+GP+Bhattacharjee%2E+2001%2E+Temporal+representation+and+reasoning+in+artificial+intelligence%3A+A+review%2E+Mathematical+and+Computer+Modelling%2C+34%281%2D2%29%3A55%E2%80%9380%2E)
36. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. [find](https://scholar.google.com/scholar?q=Jeffrey+Pennington%2C+Richard+Socher%2C+and+Christopher+D%2E+Manning%2E+2014%2E+Glove%3A+Global+vectors+for+word+representation%2E+In+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2C+pages+1532%E2%80%931543%2E)
37. Adam Poliak, Aparajita Haldar, Rachel Rudinger, J. Edward Hu, Ellie Pavlick, Aaron Steven White, and Benjamin Van Durme. 2018. Collecting diverse natural language inference problems for sentence representation evaluation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 67–81, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Adam+Poliak%2C+Aparajita+Haldar%2C+Rachel+Rudinger%2C+J%2E+Edward+Hu%2C+Ellie+Pavlick%2C+Aaron+Steven+White%2C+and+Benjamin+Van+Durme%2E+2018%2E+Collecting+diverse+natural+language+inference+problems+for+sentence+repre)
38. James Pustejovsky, Patrick Hanks, Roser Sauri, Andrew See, Robert Gaizauskas, Andrea Setzer, Dragomir Radev, Beth Sundheim, David Day, Lisa Ferro, et al. 2003. The timebank corpus. In Corpus linguistics, volume 2003, page 40. Lancaster, UK. [find](https://scholar.google.com/scholar?q=James+Pustejovsky%2C+Patrick+Hanks%2C+Roser+Sauri%2C+Andrew+See%2C+Robert+Gaizauskas%2C+Andrea+Setzer%2C+Dragomir+Radev%2C+Beth+Sundheim%2C+David+Day%2C+Lisa+Ferro%2C+et+al%2E+2003%2E+The+timebank+corpus%2E+In+Corpus+linguisti)
39. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D Manning. 2020. Stanza: A python natural language processing toolkit for many human languages. arXiv preprint arXiv:2003.07082. [↗](https://arxiv.org/abs/2003.07082) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2020-qi-stanza-python-natural-language-processing/)
40. Gabriel A Radvansky and Jeffrey M Zacks. 2014. Event cognition. Oxford University Press. [find](https://scholar.google.com/scholar?q=Gabriel+A+Radvansky+and+Jeffrey+M+Zacks%2E+2014%2E+Event+cognition%2E+Oxford+University+Press%2E)
41. Nils Reimers, Nazanin Dehghani, and Iryna Gurevych. 2016. Temporal anchoring of events for the timebank corpus. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2195– 2204. [find](https://scholar.google.com/scholar?q=Nils+Reimers%2C+Nazanin+Dehghani%2C+and+Iryna+Gurevych%2E+2016%2E+Temporal+anchoring+of+events+for+the+timebank+corpus%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Linguisti)
42. Alexis Ross and Ellie Pavlick. 2019. How well do NLI models capture verb veridicality? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2230–2240, Hong Kong, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Alexis+Ross+and+Ellie+Pavlick%2E+2019%2E+How+well+do+NLI+models+capture+verb+veridicality%3F+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+the+9th+Internation) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ross-2019-how-well-do-nli-models/)
43. Lenhart Schubert. 2002. Can we derive general world knowledge from texts? In Proceedings of the second international conference on Human Language Technology Research, pages 94–97. Morgan Kaufmann Publishers Inc. [find](https://scholar.google.com/scholar?q=Lenhart+Schubert%2E+2002%2E+Can+we+derive+general+world+knowledge+from+texts%3F+In+Proceedings+of+the+second+international+conference+on+Human+Language+Technology+Research%2C+pages+94%E2%80%9397%2E+Morgan+Kaufmann+Publ)
44. John Sylak-Glassman, Christo Kirov, David Yarowsky, and Roger Que. 2015. A language-independent feature schema for inflectional morphology. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 674–680. [find](https://scholar.google.com/scholar?q=John+Sylak%2DGlassman%2C+Christo+Kirov%2C+David+Yarowsky%2C+and+Roger+Que%2E+2015%2E+A+language%2Dindependent+feature+schema+for+inflectional+morphology%2E+In+Proceedings+of+the+53rd+Annual+Meeting+of+the+Association)
45. Niket Tandon, Bhavana Dalvi Mishra, Joel Grus, Wen-tau Yih, Antoine Bosselut, and Peter Clark. 2018. Reasoning about actions and state changes by injecting commonsense knowledge. arXiv preprint arXiv:1808.10012. [↗](https://arxiv.org/abs/1808.10012)
46. Julien Tourille, Olivier Ferret, Aurélie Névéol, and Xavier Tannier. 2017. Neural architecture for temporal relation extraction: A bi-LSTM approach for detecting narrative containers. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 224–230, Vancouver, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Julien+Tourille%2C+Olivier+Ferret%2C+Aur%C3%A9lie+N%C3%A9v%C3%A9ol%2C+and+Xavier+Tannier%2E+2017%2E+Neural+architecture+for+temporal+relation+extraction%3A+A+bi%2DLSTM+approach+for+detecting+narrative+containers%2E+In+Proceedings+o)
47. Naushad UzZaman, Hector Llorens, James Allen, Leon Derczynski, Marc Verhagen, and James Pustejovsky. 2013. SemEval-2013 Task 1: TempEval-3: Evaluating Time Expressions, Events, and Temporal Relations. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages 1–9, Atlanta, Georgia. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Naushad+UzZaman%2C+Hector+Llorens%2C+James+Allen%2C+Leon+Derczynski%2C+Marc+Verhagen%2C+and+James+Pustejovsky%2E+2013%2E+SemEval%2D2013+Task+1%3A+TempEval%2D3%3A+Evaluating+Time+Expressions%2C+Events%2C+and+Temporal+Relations%2E)
48. Benjamin D Van Durme. 2011. Extracting implicit knowledge from text, proquest. [find](https://scholar.google.com/scholar?q=Benjamin+D+Van+Durme%2E+2011%2E+Extracting+implicit+knowledge+from+text%2C+proquest%2E)
49. Siddharth Vashishtha, Benjamin Van Durme, and Aaron Steven White. 2019. Fine-grained temporal relation extraction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2906–2919, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Siddharth+Vashishtha%2C+Benjamin+Van+Durme%2C+and+Aaron+Steven+White%2E+2019%2E+Fine%2Dgrained+temporal+relation+extraction%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Lingui)
50. Chengyu Wang, Xiaofeng He, and Aoying Zhou. 2018. Event phase oriented news summarization. World Wide Web, 21(4):1069–1092. [find](https://scholar.google.com/scholar?q=Chengyu+Wang%2C+Xiaofeng+He%2C+and+Aoying+Zhou%2E+2018%2E+Event+phase+oriented+news+summarization%2E+World+Wide+Web%2C+21%284%29%3A1069%E2%80%931092%2E)
51. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112–1122. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Nikita+Nangia%2C+and+Samuel+Bowman%2E+2018%2E+A+broad%2Dcoverage+challenge+corpus+for+sentence+understanding+through+inference%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chap)
52. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, R’emi Louf, Morgan Funtow-icz, and Jamie Brew. 2019. Huggingface’s transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771. [↗](https://arxiv.org/abs/1910.03771)
53. Jeffrey M Zacks and Barbara Tversky. 2001. Event structure in perception and conception. Psychological bulletin, 127(1):3. [find](https://scholar.google.com/scholar?q=Jeffrey+M+Zacks+and+Barbara+Tversky%2E+2001%2E+Event+structure+in+perception+and+conception%2E+Psychological+bulletin%2C+127%281%29%3A3%2E)
54. Sheng Zhang, Rachel Rudinger, Kevin Duh, and Benjamin Van Durme. 2017a. Ordinal common-sense inference. Transactions of the Association of Computational Linguistics, 5(1):379–395. [find](https://scholar.google.com/scholar?q=Sheng+Zhang%2C+Rachel+Rudinger%2C+Kevin+Duh%2C+and+Benjamin+Van+Durme%2E+2017a%2E+Ordinal+common%2Dsense+inference%2E+Transactions+of+the+Association+of+Computational+Linguistics%2C+5%281%29%3A379%E2%80%93395%2E)
55. Sheng Zhang, Rachel Rudinger, and Benjamin Van Durme. 2017b. An evaluation of predpatt and open ie via stage 1 semantic role labeling. In IWCS 2017—12th International Conference on Computational Semantics (Short papers). [find](https://scholar.google.com/scholar?q=Sheng+Zhang%2C+Rachel+Rudinger%2C+and+Benjamin+Van+Durme%2E+2017b%2E+An+evaluation+of+predpatt+and+open+ie+via+stage+1+semantic+role+labeling%2E+In+IWCS+2017%E2%80%9412th+International+Conference+on+Computational+Seman)
56. Ben Zhou, Daniel Khashabi, Qiang Ning, and Dan Roth. 2019. “going on a vacation” takes longer than “going for a walk”: A study of temporal commonsense understanding. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3363–3369, Hong Kong, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ben+Zhou%2C+Daniel+Khashabi%2C+Qiang+Ning%2C+and+Dan+Roth%2E+2019%2E+%E2%80%9Cgoing+on+a+vacation%E2%80%9D+takes+longer+than+%E2%80%9Cgoing+for+a+walk%E2%80%9D%3A+A+study+of+temporal+commonsense+understanding%2E+In+Proceedings+of+the+2019+Confere)
57. Ben Zhou, Qiang Ning, Daniel Khashabi, and Dan Roth. 2020. Temporal common sense acquisition with minimal supervision. arXiv preprint arXiv:2005.04304. [↗](https://arxiv.org/abs/2005.04304)
58. Aaron Steven White, Pushpendre Rastogi, Kevin Duh, and Benjamin Van Durme. 2017. Inference is everything: Recasting semantic resources into a unified evaluation framework. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 996–1005, Taipei, Taiwan. Asian Federation of Natural Language Processing. [find](https://scholar.google.com/scholar?q=Aaron+Steven+White%2C+Pushpendre+Rastogi%2C+Kevin+Duh%2C+and+Benjamin+Van+Durme%2E+2017%2E+Inference+is+everything%3A+Recasting+semantic+resources+into+a+unified+evaluation+framework%2E+In+Proceedings+of+the+Eighth) [our card](https://md.mova.elamur.ai/free/en/docs/papers/white-2017-inference-is-everything-recasting-semantic/)
59. Aaron Steven White, Drew Reisinger, Keisuke Sakaguchi, Tim Vieira, Sheng Zhang, Rachel Rudinger, Kyle Rawlins, and Benjamin Van Durme. 2016. Universal decompositional semantics on universal dependencies. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1713–1723, Austin, TX. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Aaron+Steven+White%2C+Drew+Reisinger%2C+Keisuke+Sakaguchi%2C+Tim+Vieira%2C+Sheng+Zhang%2C+Rachel+Rudinger%2C+Kyle+Rawlins%2C+and+Benjamin+Van+Durme%2E+2016%2E+Universal+decompositional+semantics+on+universal+dependenci) [our card](https://md.mova.elamur.ai/free/en/docs/papers/white-2016-universal-decompositional-semantics-universal-dependencies/)