[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/kyle-2022-dependency-treebank-spoken-second-language/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / A Dependency Treebank of Spoken Second Language English

# A Dependency Treebank of Spoken Second Language English

**Authors:** [Kristopher Kyle](https://md.mova.elamur.ai/free/en/docs/papers/authors/kristopher-kyle/), Masaki Eguchi, Aaron Miller, Theodore Sither · **Year:** 2022 · **Venue:** Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022) **Link:** [aclanthology.org/2022.bea-1.7](https://aclanthology.org/2022.bea-1.7/) (ACL Anthology 2022.bea-1.7; DOI [10.18653/v1/2022.bea-1.7](https://doi.org/10.18653/v1/2022.bea-1.7)) **License of the paper:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) (ACL Anthology) — [creativecommons.org/licenses/by/4.0](https://creativecommons.org/licenses/by/4.0/)

[**11** citations](https://openalex.org/works?filter=cites:W4287854641) openalex · 2026-10-18[**16** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/kyle-2022-dependency-treebank-spoken-second-language/) of the 140673 papers whose bibliographies we read[Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022)](https://aclanthology.org/volumes/2022.bea-1/)[licence](https://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://aclanthology.org/2022.bea-1.7/)
- [PDF](https://aclanthology.org/2022.bea-1.7.pdf)
- [DOI 10.18653/v1/2022.bea-1.7](https://doi.org/10.18653/v1/2022.bea-1.7)
- [OpenAlex](https://openalex.org/W4287854641)

**Authors (4)**

1. [Kristopher Kyle](https://openalex.org/A5026166820) [ORCID](https://orcid.org/0000-0001-5415-9672) · h 31 · 5076 citations · 88 works
2. [Masaki Eguchi](https://openalex.org/A5019596356) [ORCID](https://orcid.org/0000-0002-0791-4935) · h 13 · 1061 citations · 47 works
3. Aaron Miller
4. [Theodore Sither](https://openalex.org/A5023308205) h 1 · 11 citations · 1 works

Citations and author metrics from openalex (2026-10-18); authors with the same name are not always told apart.

## Summary

POS taggers and parsers lose accuracy on text unlike their training data, and second-language (L2) speech is a clear case; a written L2 treebank existed, but no freely available spoken one. The authors build SL2E from utterances of Japanese learners in oral proficiency interviews (NICT JLE corpus), with interviewer turns removed. Sentences are hand-annotated with Penn POS tags and UD dependencies, each by at least two annotators with a third adjudicating disagreements. They then train spaCy models (RoBERTa-base) on L1 data only, on L1 plus L2, and on L1 plus L2 at equal total size. Even a small amount of L2 data improves both tagging and parsing markedly, with the largest gains on spoken L2 (LAS on SL2E rising from 0.876 to about 0.935). The resource is aimed at language-development research and educational NLP.

## How Mova uses it

- [Text with errors is annotated literally](https://md.mova.elamur.ai/free/en/docs/en/seeds/errors/literal-annotation-learner/) — cited as evidence that learner language needs in-domain data (a little L2 data lifts LAS from 0.876 to 0.935), in the note establishing that Mova annotates erroneous text literally (what is written, not what was meant).
- Background reading only; SL2E is not used as training or evaluation data in Mova.

## Effectiveness in Mova

Background reading for one seed note. Not measured separately.

---

👨‍🔬💥

Topics: [Treebanks](https://md.mova.elamur.ai/free/en/docs/papers/topics/treebanks/) · [Part-of-speech tagging](https://md.mova.elamur.ai/free/en/docs/papers/topics/part-of-speech-tagging/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.18653/v1/2022.bea-1.7](https://doi.org/10.18653/v1/2022.bea-1.7)

```
@inproceedings{kyle2022dependency,
  title = {{A Dependency Treebank of Spoken Second Language English}},
  author = {Kyle, Kristopher and Eguchi, Masaki and Miller, Aaron and Sither, Theodore},
  booktitle = {Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022)},
  year = {2022},
  doi = {10.18653/v1/2022.bea-1.7},
  url = {https://doi.org/10.18653/v1/2022.bea-1.7}
}
```

- [← previous A Handbook of Present-Day English](https://md.mova.elamur.ai/free/en/docs/papers/kruisinga-1925-handbook-present-day-english/)
- [next → Hedges: A Study in Meaning Criteria and the Logic of Fuzzy Concepts](https://md.mova.elamur.ai/free/en/docs/papers/lakoff-1973-hedges-study-meaning-criteria/)
- [↑ back to papers](https://md.mova.elamur.ai/free/en/docs/papers/)

## References

32 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Granger, S., & Bestgen, Y. (2014). The use of collocations by intermediate vs. Advanced non-native writers: A bigram-based study. [find](https://scholar.google.com/scholar?q=Granger%2C+S%2E%2C+%26+Bestgen%2C+Y%2E+%282014%29%2E+The+use+of+collocations+by+intermediate+vs%2E+Advanced+non%2Dnative+writers%3A+A+bigram%2Dbased+study%2E)
2. International Anthony, L. (2019). AntConc (3.5.8) [Computer software]. Waseda University. Review of Applied Linguistics in Language Teaching, 52(3), 229–252. International Journal of Corpus Linguistics, 15(4), 474–496. https://doi.org/10.1075/ijcl.15.4.02lu [↗](https://doi.org/10.1075/ijcl.15.4.02lu)
3. Grant, L., & Ginther, A. (2000). Using computer-tagged linguistic features to describe L2 writing differences. Journal of Second Language Writing, 9(2), 123–145. [find](https://scholar.google.com/scholar?q=Grant%2C+L%2E%2C+%26+Ginther%2C+A%2E+%282000%29%2E+Using+computer%2Dtagged+linguistic+features+to+describe+L2+writing+differences%2E+Journal+of+Second+Language+Writing%2C+9%282%29%2C+123%E2%80%93145%2E)
4. Lu, X. (2011). A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers’ language development. TESOL Quarterly, 45(1), 36–62. [find](https://scholar.google.com/scholar?q=Lu%2C+X%2E+%282011%29%2E+A+corpus%2Dbased+evaluation+of+syntactic+complexity+measures+as+indices+of+college%2Dlevel+ESL+writers%E2%80%99+language+development%2E+TESOL+Quarterly%2C+45%281%29%2C+36%E2%80%9362%2E)
5. Honnibal, M., Montani, I., Van Landeghem, S., & Boyd, A. (2020). spaCy: Industrial-strength Natural Language Processing in Python [Python]. https://doi.org/10.5281/zenodo.1212303 (Original work published 2014) [↗](https://doi.org/10.5281/zenodo.1212303)
6. Martens, S. (2013). TüNDRA: A web application for treebank search and visualization. 133. [find](https://scholar.google.com/scholar?q=Martens%2C+S%2E+%282013%29%2E+T%C3%BCNDRA%3A+A+web+application+for+treebank+search+and+visualization%2E+133%2E)
7. McCarthy, P. M., & Jarvis, S. (2010). MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods, 42(2), 381–392. https://doi.org/10.3758/BRM.42.2.381 [↗](https://doi.org/10.3758/brm.42.2.381)
8. Izumi, E., Uchimoto, K., & Isahara, H. (2004). The NICT JLE Corpus: Exploiting the language learners’ speech database for research and education. International Journal of The Computer, the Internet and Management, 12(2), 119–125. [find](https://scholar.google.com/scholar?q=Izumi%2C+E%2E%2C+Uchimoto%2C+K%2E%2C+%26+Isahara%2C+H%2E+%282004%29%2E+The+NICT+JLE+Corpus%3A+Exploiting+the+language+learners%E2%80%99+speech+database+for+research+and+education%2E+International+Journal+of+The+Computer%2C+the+Internet+an)
9. McClosky, D., Charniak, E., & Johnson, M. (2006). Reranking and self-training for parser adaptation. 337–344. [find](https://scholar.google.com/scholar?q=McClosky%2C+D%2E%2C+Charniak%2C+E%2E%2C+%26+Johnson%2C+M%2E+%282006%29%2E+Reranking+and+self%2Dtraining+for+parser+adaptation%2E+337%E2%80%93344%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2006-mcclosky-reranking-self-training-parser-adaptation/)
10. Jarvis, S., & Hashimoto, B. J. (2021). How operationalizations of word types affect measures of lexical diversity. International Journal of Learner Corpus Research, 7(1), 163–194. [find](https://scholar.google.com/scholar?q=Jarvis%2C+S%2E%2C+%26+Hashimoto%2C+B%2E+J%2E+%282021%29%2E+How+operationalizations+of+word+types+affect+measures+of+lexical+diversity%2E+International+Journal+of+Learner+Corpus+Research%2C+7%281%29%2C+163%E2%80%93194%2E)
11. Meurers, D., & Dickinson, M. (2017). Evidence and interpretation in language learning research: Opportunities for collaboration with computational linguistics. Language Learning, 67(S1), 66–95. [find](https://scholar.google.com/scholar?q=Meurers%2C+D%2E%2C+%26+Dickinson%2C+M%2E+%282017%29%2E+Evidence+and+interpretation+in+language+learning+research%3A+Opportunities+for+collaboration+with+computational+linguistics%2E+Language+Learning%2C+67%28S1%29%2C+66%E2%80%9395%2E)
12. Kyle, K. (2021). Natural language processing for learner corpus research. International Journal of Learner Corpus Research, 7(1), 1–16. [find](https://scholar.google.com/scholar?q=Kyle%2C+K%2E+%282021%29%2E+Natural+language+processing+for+learner+corpus+research%2E+International+Journal+of+Learner+Corpus+Research%2C+7%281%29%2C+1%E2%80%9316%2E)
13. Monarch, R. M. (2021). Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI. Simon and Schuster. [find](https://scholar.google.com/scholar?q=Monarch%2C+R%2E+M%2E+%282021%29%2E+Human%2Din%2Dthe%2DLoop+Machine+Learning%3A+Active+learning+and+annotation+for+human%2Dcentered+AI%2E+Simon+and+Schuster%2E)
14. Kyle, K., & Crossley, S. A. (2017). Assessing syntactic sophistication in L2 writing: A usage-based approach. Language Testing, 34(4), 513–535. [find](https://scholar.google.com/scholar?q=Kyle%2C+K%2E%2C+%26+Crossley%2C+S%2E+A%2E+%282017%29%2E+Assessing+syntactic+sophistication+in+L2+writing%3A+A+usage%2Dbased+approach%2E+Language+Testing%2C+34%284%29%2C+513%E2%80%93535%2E)
15. Nagata, R. & Sakaguchi, K. (2016). Phrase Structure Annotation and Parsing for Learner English. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1837–1847, Berlin, Germany. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Nagata%2C+R%2E+%26+Sakaguchi%2C+K%2E+%282016%29%2E+Phrase+Structure+Annotation+and+Parsing+for+Learner+English%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1%3A+Lo)
16. Kyle, K., & Crossley, S. A. (2018). Measuring Syntactic Complexity in L2 Writing Using Fine-Grained Clausal and Phrasal Indices. The Modern Language Journal, 102(2), 333–349. https://doi.org/10.1111/modl.12468 [↗](https://doi.org/10.1111/modl.12468)
17. Kyle, K., Crossley, S. A., & Berger, C. M. (2018). The tool for the automatic analysis of lexical sophistication (TAALES): Version 2.0. Behavior Research Methods, 50(3), 1030–1046. https://doi.org/10.3758/s13428-017-0924-4 [↗](https://doi.org/10.3758/s13428-017-0924-4)
18. Nivre, J., Marneffe, M.-C. de, Ginter, F., Hajic, J., Manning, C. D., Pyysalo, S., Schuster, S., Tyers, F., & Zeman, D. (2020). Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection. 4034–4043. https://aclanthology.org/2020.lrec-1.497 [↗](https://aclanthology.org/2020.lrec-1.497/) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2020-nivre-universal-dependencies-v2-evergrowing-multilingual/)
19. Kyle, K., Crossley, S., & Verspoor, M. (2021). Measuring longitudinal writing development using indices of syntactic complexity and sophistication. Studies in Second Language Acquisition, 43(4), 781–812. [find](https://scholar.google.com/scholar?q=Kyle%2C+K%2E%2C+Crossley%2C+S%2E%2C+%26+Verspoor%2C+M%2E+%282021%29%2E+Measuring+longitudinal+writing+development+using+indices+of+syntactic+complexity+and+sophistication%2E+Studies+in+Second+Language+Acquisition%2C+43%284%29%2C+781%E2%80%938)
20. Paquot, M. (2018). Phraseological Competence: A Missing Component in University Entrance Language Tests? Insights From a Study of EFL Learners’ Use of Statistical Collocations. Language Assessment Quarterly, 15(1), 29–43. [find](https://scholar.google.com/scholar?q=Paquot%2C+M%2E+%282018%29%2E+Phraseological+Competence%3A+A+Missing+Component+in+University+Entrance+Language+Tests%3F+Insights+From+a+Study+of+EFL+Learners%E2%80%99+Use+of+Statistical+Collocations%2E+Language+Assessment+Qua)
21. Kyle, K., & Eguchi, M. (2021). Automatically assessing lexical sophistication using word, bigram, and dependency indices. In Perspectives on the L2 Phrasicon: The View from Learner Corpora (pp. 126–151). Multilingual Matters. [find](https://scholar.google.com/scholar?q=Kyle%2C+K%2E%2C+%26+Eguchi%2C+M%2E+%282021%29%2E+Automatically+assessing+lexical+sophistication+using+word%2C+bigram%2C+and+dependency+indices%2E+In+Perspectives+on+the+L2+Phrasicon%3A+The+View+from+Learner+Corpora+%28pp%2E+126%E2%80%9315)
22. Paquot, M. (2019). The phraseological dimension in interlanguage complexity research. Second Language Research, 35(1), 121–145. [find](https://scholar.google.com/scholar?q=Paquot%2C+M%2E+%282019%29%2E+The+phraseological+dimension+in+interlanguage+complexity+research%2E+Second+Language+Research%2C+35%281%29%2C+121%E2%80%93145%2E)
23. Polio, C., & Yoon, H. (2018). The reliability and validity of automated tools for examining variation in syntactic complexity across genres. International Journal of Applied Linguistics, 28(1), 165–188. [find](https://scholar.google.com/scholar?q=Polio%2C+C%2E%2C+%26+Yoon%2C+H%2E+%282018%29%2E+The+reliability+and+validity+of+automated+tools+for+examining+variation+in+syntactic+complexity+across+genres%2E+International+Journal+of+Applied+Linguistics%2C+28%281%29%2C+165%E2%80%9318)
24. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach (arXiv:1907.11692). arXiv. http://arxiv.org/abs/1907.11692 [↗](https://arxiv.org/abs/1907.11692)
25. Rubin, R. (2021). Assessing the impact of automatic dependency annotation on the measurement of phraseological complexity in L2 Dutch. International Journal of Learner Corpus Research, 7(1), 131–162. [find](https://scholar.google.com/scholar?q=Rubin%2C+R%2E+%282021%29%2E+Assessing+the+impact+of+automatic+dependency+annotation+on+the+measurement+of+phraseological+complexity+in+L2+Dutch%2E+International+Journal+of+Learner+Corpus+Research%2C+7%281%29%2C+131%E2%80%93162%2E)
26. Lu, X. (2010). Automatic analysis of syntactic complexity in second language writing. [find](https://scholar.google.com/scholar?q=Lu%2C+X%2E+%282010%29%2E+Automatic+analysis+of+syntactic+complexity+in+second+language+writing%2E)
27. Sakaguchi, K., Post, M., and Van Durme, B. (2017). Error-repair Dependency Parsing for Ungrammatical Texts. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 189–195, Vancouver, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sakaguchi%2C+K%2E%2C+Post%2C+M%2E%2C+and+Van+Durme%2C+B%2E+%282017%29%2E+Error%2Drepair+Dependency+Parsing+for+Ungrammatical+Texts%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28)
28. Santorini, B. (1990). Part-of-speech tagging guidelines for the Penn Treebank project (3rd revision, 2nd printing). Ms., Department of Linguistics, UPenn. Philadelphia, PA. [find](https://scholar.google.com/scholar?q=Santorini%2C+B%2E+%281990%29%2E+Part%2Dof%2Dspeech+tagging+guidelines+for+the+Penn+Treebank+project+%283rd+revision%2C+2nd+printing%29%2E+Ms%2E%2C+Department+of+Linguistics%2C+UPenn%2E+Philadelphia%2C+PA%2E) [our card](https://md.mova.elamur.ai/free/en/docs/papers/santorini-1990-part-of-speech-tagging-guidelines-penn-treebank/)
29. Silveira, N., Dozat, T., De Marneffe, M.-C., Bowman, S., Connor, M., Bauer, J., & Manning, C. (2014). A Gold Standard Dependency Corpus for English. Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), 2897–2904. [find](https://scholar.google.com/scholar?q=Silveira%2C+N%2E%2C+Dozat%2C+T%2E%2C+De+Marneffe%2C+M%2E%2DC%2E%2C+Bowman%2C+S%2E%2C+Connor%2C+M%2E%2C+Bauer%2C+J%2E%2C+%26+Manning%2C+C%2E+%282014%29%2E+A+Gold+Standard+Dependency+Corpus+for+English%2E+Proceedings+of+the+Ninth+International+Conference+o)
30. Yannakoudakis, H., Briscoe, T., & Medlock, B. (2011). A new dataset and method for automatically grading ESOL texts. 180–189. [find](https://scholar.google.com/scholar?q=Yannakoudakis%2C+H%2E%2C+Briscoe%2C+T%2E%2C+%26+Medlock%2C+B%2E+%282011%29%2E+A+new+dataset+and+method+for+automatically+grading+ESOL+texts%2E+180%E2%80%93189%2E)
31. Zeldes, A. (2017). The GUM corpus: Creating multilayer resources in the classroom. Language Resources and Evaluation, 51(3), 581–612. [find](https://scholar.google.com/scholar?q=Zeldes%2C+A%2E+%282017%29%2E+The+GUM+corpus%3A+Creating+multilayer+resources+in+the+classroom%2E+Language+Resources+and+Evaluation%2C+51%283%29%2C+581%E2%80%93612%2E)
32. Zeman, D., Popel, M., Straka, M., Hajic, J., Nivre, J., Ginter, F., Luotolahti, J., Pyysalo, S., Petrov, S., & Potthast, M. (2017). CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies. 1–19. [find](https://scholar.google.com/scholar?q=Zeman%2C+D%2E%2C+Popel%2C+M%2E%2C+Straka%2C+M%2E%2C+Hajic%2C+J%2E%2C+Nivre%2C+J%2E%2C+Ginter%2C+F%2E%2C+Luotolahti%2C+J%2E%2C+Pyysalo%2C+S%2E%2C+Petrov%2C+S%2E%2C+%26+Potthast%2C+M%2E+%282017%29%2E+CoNLL+2017+shared+task%3A+Multilingual+parsing+from+raw+text+to+univer) [our card](https://md.mova.elamur.ai/free/en/docs/papers/ud-2017-zeman-conll-2017-shared-task-multilingual/)