[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/findings-2022-kulick-penn-helsinki-parsed-corpus-early/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / Penn-Helsinki Parsed Corpus of Early Modern English: First Parsing Results and Analysis

# Penn-Helsinki Parsed Corpus of Early Modern English: First Parsing Results and Analysis

**Authors:** [Seth Kulick](https://md.mova.elamur.ai/free/en/docs/papers/authors/seth-kulick/), [Neville Ryant](https://md.mova.elamur.ai/free/en/docs/papers/authors/neville-ryant/) and [Beatrice Santorini](https://md.mova.elamur.ai/free/en/docs/papers/authors/beatrice-santorini/) · **Year:** 2022 · **Venue:** Findings of the Association for Computational Linguistics: NAACL 2022 **Link:** [aclanthology.org/2022.findings-naacl.44](https://aclanthology.org/2022.findings-naacl.44/) · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) (ACL Anthology); the PPCEME data is licensed for research — tests only

[**2** citations](https://openalex.org/works?filter=cites:W4287887670) openalex-snapshot · 2026-10-14[**5** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/findings-2022-kulick-penn-helsinki-parsed-corpus-early/) of the 140673 papers whose bibliographies we read[Findings of the Association for Computational Linguistics: NAACL 2022](https://aclanthology.org/volumes/2022.findings-naacl/)[licence](https://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://aclanthology.org/2022.findings-naacl.44/)
- [PDF](https://aclanthology.org/2022.findings-naacl.44.pdf)
- [DOI 10.18653/v1/2022.findings-naacl.44](https://doi.org/10.18653/v1/2022.findings-naacl.44)
- [OpenAlex](https://openalex.org/W4287887670)

**Authors (3)**

1. [Seth Kulick](https://openalex.org/A5090150146) [ORCID](https://orcid.org/0000-0002-1376-4902) · [Semantic Scholar](https://www.semanticscholar.org/author/2695965) · h 15 · 1114 citations · 139 works
2. [Neville Ryant](https://openalex.org/A5088320958) [ORCID](https://orcid.org/0000-0003-2545-6912) · [Semantic Scholar](https://www.semanticscholar.org/author/2980165) · h 19 · 1969 citations · 111 works
3. [Beatrice Santorini](https://openalex.org/A5023146320) [Semantic Scholar](https://www.semanticscholar.org/author/2424234) · h 14 · 2138 citations · 47 works

Citations and author metrics from openalex-snapshot (2026-10-14); authors with the same name are not always told apart.

## What it is about

The PPCEME is a 1.7-million-word treebank of English from 1500 to 1712. It has a much larger POS set than the Penn Treebank and uses function tags (SBJ, OB1, MAT, THT and others) where the PTB uses tree structure. The authors train a self-attentive constituency parser on eight cross-validation splits, with four kinds of embeddings. ELMo trained on EEBO gives the best F1, about 90.5 on test with 31 function tags, 0.67 points above BERT. Function-tag scores fall from 97.90 with 10 tags to 95.55 with 31 on test. The direct speech tag SPE, at about 4% of the tokens, scores 46.08 F1, because the source has no quotation marks. The label vocabulary grows from 250 to 938 labels with 31 tags, and only about 279 of the training labels occur in test.

## Ideas

- I1. `en/src/parse.rs`: UD relation subtypes (such as `nsubj:pass` or `obl:tmod`) multiply the labels as the function tags do here. Split the work into the base relation, as now, and a subtype stage that runs after it, which the paper proposes as future work. Report F1 per subtype with its frequency, measured on EWT test.
- I2. Label-vocabulary audit in training: count the relation and subtype labels that occur in train but not in dev or test, and those in test but not in train, and report how much LAS is lost on the unseen ones. Two runs must agree.
- I3. Imperatives: the paper shows a bare base-form verb that is either imperative or present tense, confused by the parser when the subject is missing. Test a rule in the tag fixes: a sentence-initial bare verb under the root, with no `nsubj`, gets `Mood=Imp`. Check precision by hand on 50 EWT test sentences, with sentences that have an explicit subject as the negative control, where the rule must not fire.
- I4. Direct speech needs context across sentences, as the paper says. For the tale lessons, use quotation marks and the speaker references from `coref` to mark sentences inside a quote, and measure on a hand-checked sample of quoted sentences.

## For Mova

The chapters are [09. Early Modern and archaic forms](https://md.mova.elamur.ai/free/en/docs/languages/en/09-early-modern-and-archaic/) (the early register), [13. Verbs](https://md.mova.elamur.ai/free/en/docs/languages/en/13-verbs/) (imperatives), [20. Syntax: the Universal Dependencies frame](https://md.mova.elamur.ai/free/en/docs/languages/en/20-syntax-overview/) (label sets) and [38. Dialogue, discourse markers and reported speech](https://md.mova.elamur.ai/free/en/docs/languages/en/38-dialogue-and-discourse-markers/) (quotes). The code is `en/src/parse.rs` and `coref`. The PPCEME data is for research: use it for tests only. Any EEBO-based pretraining would need a licence check in [Data and model licences](https://md.mova.elamur.ai/free/en/docs/licence/DATA/) first.

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.18653/v1/2022.findings-naacl.44](https://doi.org/10.18653/v1/2022.findings-naacl.44)

```
@inproceedings{kulick2022penn,
  title = {{Penn-Helsinki Parsed Corpus of Early Modern English: First Parsing Results and Analysis}},
  author = {Kulick, Seth and Ryant, Neville and Santorini, Beatrice},
  booktitle = {Findings of the Association for Computational Linguistics: NAACL 2022},
  year = {2022},
  doi = {10.18653/v1/2022.findings-naacl.44},
  url = {https://doi.org/10.18653/v1/2022.findings-naacl.44}
}
```

- [← previous A Closer Look into the Robustness of Neural Dependency Parsers Using Better Adversarial Examples](https://md.mova.elamur.ai/free/en/docs/papers/findings-2021-wang-closer-look-into-robustness-neural/)
- [next → Large-Scale QA-SRL Parsing](https://md.mova.elamur.ai/free/en/docs/papers/fitzgerald-2018-large-scale-qa-srl-parsing/)
- [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/)

## References

48 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Dogu Araci. 2019. FinBERT: Financial sentiment analysis with pre-trained language models. arXiv preprint arXiv:1908.10063. [↗](https://arxiv.org/abs/1908.10063)
2. Iz Beltagy, Arman Cohan, and Kyle Lo. 2019. SciB-ERT: A pretrained language model for scientific text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). [find](https://scholar.google.com/scholar?q=Iz+Beltagy%2C+Arman+Cohan%2C+and+Kyle+Lo%2E+2019%2E+SciB%2DERT%3A+A+pretrained+language+model+for+scientific+text%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+the)
3. Don Blaheta. 2003. Function Tagging. Ph.D. thesis, Brown University. [find](https://scholar.google.com/scholar?q=Don+Blaheta%2E+2003%2E+Function+Tagging%2E+Ph%2ED%2E+thesis%2C+Brown+University%2E)
4. Don Blaheta and Eugene Charniak. 2000. Assigning function tags to parsed text. In 1st Meeting of the North American Chapter of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Don+Blaheta+and+Eugene+Charniak%2E+2000%2E+Assigning+function+tags+to+parsed+text%2E+In+1st+Meeting+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%2E)
5. Shu Cai, David Chiang, and Yoav Goldberg. 2011. Language-independent parsing with empty elements. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 212–216, Portland, Oregon, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Shu+Cai%2C+David+Chiang%2C+and+Yoav+Goldberg%2E+2011%2E+Language%2Dindependent+parsing+with+empty+elements%2E+In+Proceedings+of+the+49th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%3A+Human+Lang)
6. Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, and Ion Androutsopoulos. 2020. LEGAL-BERT: The muppets straight out of law school. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2898– 2904, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ilias+Chalkidis%2C+Manos+Fergadiotis%2C+Prodromos+Malakasiotis%2C+Nikolaos+Aletras%2C+and+Ion+Androutsopoulos%2E+2020%2E+LEGAL%2DBERT%3A+The+muppets+straight+out+of+law+school%2E+In+Findings+of+the+Association+for+Comp)
7. Yufei Chen, Yuanyuan Zhao, Weiwei Sun, and Xiaojun Wan. 2018. Pre- and in-parsing models for neural empty category detection. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2687–2696, Melbourne, Australia. Association for Computational Linguistics. [↗](https://doi.org/10.60692/sxfjm-91060)
8. Qiao Jin, Bhuwan Dhingra, William W Cohen, and Xinghua Lu. 2019. Probing biomedical embeddings from language models. arXiv preprint arXiv:1904.02181. [↗](https://arxiv.org/abs/1904.02181)
9. Mark Johnson. 2002. A simple pattern-matching algorithm for recovering empty nodes and their antecedents. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 136–143, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mark+Johnson%2E+2002%2E+A+simple+pattern%2Dmatching+algorithm+for+recovering+empty+nodes+and+their+antecedents%2E+In+Proceedings+of+the+40th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pa)
10. Andres Karjus. 2020. Competition, selection and communicative need in language change: An investigation using corpora, computational modelling and experimentation. Ph.D. thesis, University of Edinburgh. [↗](https://doi.org/10.7488/era/1273)
11. Nikita Kitaev, Steven Cao, and Dan Klein. 2019. Multilingual constituency parsing with self-attention and pre-training. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pages 3499–3505. [find](https://scholar.google.com/scholar?q=Nikita+Kitaev%2C+Steven+Cao%2C+and+Dan+Klein%2E+2019%2E+Multilingual+constituency+parsing+with+self%2Dattention+and+pre%2Dtraining%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+L)
12. Nikita Kitaev and Dan Klein. 2018. Constituency parsing with a self-attentive encoder. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), pages 2676–2686. [find](https://scholar.google.com/scholar?q=Nikita+Kitaev+and+Dan+Klein%2E+2018%2E+Constituency+parsing+with+a+self%2Dattentive+encoder%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28ACL%29%2C+pages+2676%E2%80%932686)
13. Anthony Kroch. 1989. Reflexes of grammar in patterns of language change. Language Variation and Change, 1(3):199–244. [↗](https://doi.org/10.1017/s0954394500000168)
14. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 4171–4186. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+Proceedings+of+the+2019+Conference+of+the+N)
15. Anthony Kroch. 2020. Penn Parsed Corpora of Historical English (LDC2020T16). Web Download. Philadelphia: Linguistic Data Consortium. [find](https://scholar.google.com/scholar?q=Anthony+Kroch%2E+2020%2E+Penn+Parsed+Corpora+of+Historical+English+%28LDC2020T16%29%2E+Web+Download%2E+Philadelphia%3A+Linguistic+Data+Consortium%2E)
16. Aaron Ecay. 2015. A multi-step analysis of the evolution of English do-support. Ph.D. thesis, University of Pennsylvania. [find](https://scholar.google.com/scholar?q=Aaron+Ecay%2E+2015%2E+A+multi%2Dstep+analysis+of+the+evolution+of+English+do%2Dsupport%2E+Ph%2ED%2E+thesis%2C+University+of+Pennsylvania%2E)
17. Anthony Kroch, Beatrice Santorini, and Lauren Delfs. 2004. Penn-Helsinki Parsed Corpus of Early Modern English (PPCEME). CD-ROM, first edition, release 3. [http://www.ling.upenn.edu/ppche/ppche-release-2016/](http://www.ling.upenn.edu/ppche/ppche-release-2016/) PPCEME-RELEASE-3. [↗](http://www.ling.upenn.edu/ppche/ppche-release-2016/)
18. Ryan Gabbard, Seth Kulick, and Mitchell Marcus. 2006. Fully parsing the Penn Treebank. In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, pages 184– 191. [find](https://scholar.google.com/scholar?q=Ryan+Gabbard%2C+Seth+Kulick%2C+and+Mitchell+Marcus%2E+2006%2E+Fully+parsing+the+Penn+Treebank%2E+In+Proceedings+of+the+Human+Language+Technology+Conference+of+the+NAACL%2C+Main+Conference%2C+pages+184%E2%80%93+191%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/bank-2006-gabbard-fully-parsing-penn-treebank/)
19. Charlotte Galves. 2020. Relaxed Verb Second in Classical Portuguese. In Rebecca Woods and Sam Wolfe, editors, Rethinking Verb-Second, pages 368– 395. Oxford University Press. [↗](https://doi.org/10.1093/oso/9780198844303.003.0016)
20. Anthony Kroch and Beatrice Santorini. 2021. Penn-BFM Parsed Corpus of Historical French, version 1.0. [https://github.com/beatrice57/](https://github.com/beatrice57/) mcvf-plus-ppchf. [↗](https://github.com/beatrice57/)
21. Anthony Kroch, Beatrice Santorini, and Ariel Diertani. 2016. Penn Parsed Corpus of Modern British English (PPCMBE2). CD-ROM, second edition, release 1. [http://www.ling.upenn.edu/ppche/ppche-release-2016/](http://www.ling.upenn.edu/ppche/ppche-release-2016/) PPCMBE2-RELEASE-1. [↗](http://www.ling.upenn.edu/ppche/ppche-release-2016/)
22. Charlotte Galves, Aroldo Leal de Andrade, and Pablo Faria. 2017. Tycho Brahe Parsed Corpus of Historical Portuguese. [http://www.tycho.iel.unicamp.br/](http://www.tycho.iel.unicamp.br/) ~tycho/corpus/texts/psd.zip. [↗](http://www.tycho.iel.unicamp.br/)
23. Anthony Kroch, Ann Taylor, and Donald Ringe. 2000a. The Middle English verb-second constraint: A case study in language contact and language change. In Susan Herring, Lene Schoessler, and Peter van Reenen, editors, Textual parameters in older language, pages 353–391. Benjamins. [↗](https://doi.org/10.1075/cilt.195.17kro)
24. Kyle Gorman and Steven Bedrick. 2019. We need to talk about standard splits. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), pages 2786–2791. [find](https://scholar.google.com/scholar?q=Kyle+Gorman+and+Steven+Bedrick%2E+2019%2E+We+need+to+talk+about+standard+splits%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28ACL%29%2C+pages+2786%E2%80%932791%2E)
25. Anthony Kroch, Ann Taylor, and Beatrice Santorini. 2000b. Penn-Helsinki Parsed Corpus of Middle English (ppcme2). CD-ROM, second edition, release 4. [find](https://scholar.google.com/scholar?q=Anthony+Kroch%2C+Ann+Taylor%2C+and+Beatrice+Santorini%2E+2000b%2E+Penn%2DHelsinki+Parsed+Corpus+of+Middle+English+%28ppcme2%29%2E+CD%2DROM%2C+second+edition%2C+release+4%2E)
26. Seth Kulick, Anthony Kroch, and Beatrice Santorini. 2014. The Penn Parsed Corpus of Modern British English: First parsing results and analysis. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL), pages 662–667. [find](https://scholar.google.com/scholar?q=Seth+Kulick%2C+Anthony+Kroch%2C+and+Beatrice+Santorini%2E+2014%2E+The+Penn+Parsed+Corpus+of+Modern+British+English%3A+First+parsing+results+and+analysis%2E+In+Proceedings+of+the+52nd+Annual+Meeting+of+the+Associa) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2014-kulick-penn-parsed-corpus-modern-british/)
27. Seth Kulick, Neville Ryant, and Beatrice Santorini. 2022. Parsing Early Modern English for linguistic search. In Proceedings of the Society for Computation in Linguistics 2022, pages 143–157, online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Seth+Kulick%2C+Neville+Ryant%2C+and+Beatrice+Santorini%2E+2022%2E+Parsing+Early+Modern+English+for+linguistic+search%2E+In+Proceedings+of+the+Society+for+Computation+in+Linguistics+2022%2C+pages+143%E2%80%93157%2C+online%2E+) [our card](https://md.mova.elamur.ai/free/en/docs/papers/kulick-2022-parsing-early-modern-english-linguistic/)
28. Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234–1240. [↗](https://doi.org/10.1093/bioinformatics/btz682)
29. Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313–330. [↗](https://doi.org/10.21236/ada273556) [our card](https://md.mova.elamur.ai/free/en/docs/papers/bank-1993-marcus-building-large-annotated-corpus-english/)
30. France Martineau, Paul Hirschbühler, Anthony Kroch, and Yves Charles Morin. 2021. MCVF Corpus, parsed, version 2.0. [https://github.com/](https://github.com/) beatrice57/mcvf-plus-ppchf. [↗](https://github.com/)
31. Djamé Seddah, Sandra Kübler, and Reut Tsarfaty. 2014. Introducing the SPMRL 2014 shared task on parsing morphologically-rich languages. In Proceedings of the First Joint Workshop on Statistical Parsing of Morphologically Rich Languages and Syntactic Analysis of Non-Canonical Languages, pages 103– 109. [find](https://scholar.google.com/scholar?q=Djam%C3%A9+Seddah%2C+Sandra+K%C3%BCbler%2C+and+Reut+Tsarfaty%2E+2014%2E+Introducing+the+SPMRL+2014+shared+task+on+parsing+morphologically%2Drich+languages%2E+In+Proceedings+of+the+First+Joint+Workshop+on+Statistical+Parsin)
32. Satoshi Sekine and Michael Collins. 2008. [http://nlp.cs.nyu.edu/evalb/](http://nlp.cs.nyu.edu/evalb/). evalb. [↗](http://nlp.cs.nyu.edu/evalb/)
33. Mitchell Stern, Jacob Andreas, and Dan Klein. 2017. A minimal span-based neural constituency parser. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistic (ACL), pages 818–827. [find](https://scholar.google.com/scholar?q=Mitchell+Stern%2C+Jacob+Andreas%2C+and+Dan+Klein%2E+2017%2E+A+minimal+span%2Dbased+neural+constituency+parser%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Computational+Linguistic+%28ACL%29%2C+pa)
34. Ann Taylor, Arja Nurmi, Anthony Warner, Susan Pintzuk, and Terttu Nevalainen. 2006. Parsed Corpus of Early English Correspondence. Distributed through the Oxford Text Archive. [find](https://scholar.google.com/scholar?q=Ann+Taylor%2C+Arja+Nurmi%2C+Anthony+Warner%2C+Susan+Pintzuk%2C+and+Terttu+Nevalainen%2E+2006%2E+Parsed+Corpus+of+Early+English+Correspondence%2E+Distributed+through+the+Oxford+Text+Archive%2E)
35. Ann Taylor, Anthony Warner, Susan Pintzuk, and Frans Beths. 2003. York-Toronto-Helsinki Parsed Corpus of Old English Prose. Distributed by the Oxford Text archive. [find](https://scholar.google.com/scholar?q=Ann+Taylor%2C+Anthony+Warner%2C+Susan+Pintzuk%2C+and+Frans+Beths%2E+2003%2E+York%2DToronto%2DHelsinki+Parsed+Corpus+of+Old+English+Prose%2E+Distributed+by+the+Oxford+Text+archive%2E)
36. Paola Merlo and Gabriele Musillo. 2005. Accurate function parsing. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 620–627. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Paola+Merlo+and+Gabriele+Musillo%2E+2005%2E+Accurate+function+parsing%2E+In+Proceedings+of+Human+Language+Technology+Conference+and+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+620%E2%80%93)
37. Taesun Moon and Jason Baldridge. 2007. Part-of-speech tagging for Middle English through alignment and projection of parallel diachronic texts. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 390–399. [find](https://scholar.google.com/scholar?q=Taesun+Moon+and+Jason+Baldridge%2E+2007%2E+Part%2Dof%2Dspeech+tagging+for+Middle+English+through+alignment+and+projection+of+parallel+diachronic+texts%2E+In+Proceedings+of+the+2007+Joint+Conference+on+Empirical) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2007-moon-part-speech-tagging-middle-english/)
38. Khalil Mrini, Franck Dernoncourt, Quan Hung Tran, Trung Bui, Walter Chang, and Ndapa Nakashole. 2020. Rethinking self-attention: Towards interpretability in neural parsing. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 731–742. [find](https://scholar.google.com/scholar?q=Khalil+Mrini%2C+Franck+Dernoncourt%2C+Quan+Hung+Tran%2C+Trung+Bui%2C+Walter+Chang%2C+and+Ndapa+Nakashole%2E+2020%2E+Rethinking+self%2Dattention%3A+Towards+interpretability+in+neural+parsing%2E+In+Findings+of+the+Associat)
39. Sebastian Nagel. 2016. Cc-news. //commoncrawl.org/2016/10/ news-dataset-available/. [find](https://scholar.google.com/scholar?q=Sebastian+Nagel%2E+2016%2E+Cc%2Dnews%2E+%2F%2Fcommoncrawl%2Eorg%2F2016%2F10%2F+news%2Ddataset%2Davailable%2F%2E)
40. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. https: [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+Ilya+Sutskever%2C+et+al%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2E+https%3A)
41. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 2227–2237. [find](https://scholar.google.com/scholar?q=Matthew+E%2E+Peters%2C+Mark+Neumann%2C+Mohit+Iyyer%2C+Matt+Gardner%2C+Christopher+Clark%2C+Kenton+Lee%2C+and+Luke+Zettlemoyer%2E+2018%2E+Deep+contextualized+word+representations%2E+In+Proceedings+of+the+2018+Conference+o)
42. Text Creation Partnership. 2019. Early English Books Online. [https://textcreationpartnership.org/tcp-texts/eebo-tcp-early-english-books-online/](https://textcreationpartnership.org/tcp-texts/eebo-tcp-early-english-books-online/). [↗](https://textcreationpartnership.org/tcp-texts/eebo-tcp-early-english-books-online/)
43. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems (NIPS), volume 30. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81+ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+Neural+Information+Process)
44. Joel C. Wallenberg. 2016. Extraposition is disappearing. Language, 92(4):e237–e256. [↗](https://doi.org/10.1353/lan.2016.0079)
45. Joel C. Wallenberg, Rachael Bailes, Christine Cuskley, and Anton Karl Ingason. 2021. Smooth signals and syntactic change. Languages, 6(2):60. [↗](https://doi.org/10.3390/languages6020060)
46. Joel C. Wallenberg, Anton Karl Ingason, Einar Freyr Sigurdhsson, and Eiríkur Rögnvaldsson. 2011. Icelandic Parsed Historical Corpus (IcePaHC), v0.9. [http://www.linguist.is/icelandic_](http://www.linguist.is/icelandic_) treebank. [↗](http://www.linguist.is/icelandic_)
47. Yi Yang and Jacob Eisenstein. 2016. Part-of-speech tagging for historical English. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 1318–1328. [find](https://scholar.google.com/scholar?q=Yi+Yang+and+Jacob+Eisenstein%2E+2016%2E+Part%2Dof%2Dspeech+tagging+for+historical+English%2E+In+Proceedings+of+the+2016+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%3A) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2016-yang-part-speech-tagging-historical-english/)
48. Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 19–27. [find](https://scholar.google.com/scholar?q=Yukun+Zhu%2C+Ryan+Kiros%2C+Rich+Zemel%2C+Ruslan+Salakhutdinov%2C+Raquel+Urtasun%2C+Antonio+Torralba%2C+and+Sanja+Fidler%2E+2015%2E+Aligning+books+and+movies%3A+Towards+story%2Dlike+visual+explanations+by+watching+movies+)