[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/lingbulk-2010-felt-ccash-web-application-framework-efficient/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [Aramaic](https://md.mova.elamur.ai/free/en/docs/papers/arc/) / CCASH: A Web Application Framework for Efficient, Distributed Language Resource Development

# CCASH: A Web Application Framework for Efficient, Distributed Language Resource Development

**Authors:** [Paul Felt](https://md.mova.elamur.ai/free/en/docs/papers/authors/paul-felt/); Owen Merkling; Marc Carmen; [Eric Ringger](https://md.mova.elamur.ai/free/en/docs/papers/authors/eric-ringger/); Warren Lemmon; [Kevin Seppi](https://md.mova.elamur.ai/free/en/docs/papers/authors/kevin-seppi/); [Robbie Haertel](https://md.mova.elamur.ai/free/en/docs/papers/authors/robbie-haertel/) · **Year:** 2010 · **Venue:** LREC 2010, Seventh International Conference on Language Resources and Evaluation **Link:** [aclanthology.org/L10-1248](https://aclanthology.org/L10-1248/) · library `lingbulk-2010-felt-ccash-web-application-framework-efficient` · **Licence:** [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) (ELRA, LREC proceedings) — summary only

[**7** citations](https://openalex.org/works?filter=cites:W7688137) openalex · 2026-10-19[**3** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/lingbulk-2010-felt-ccash-web-application-framework-efficient/) of the 140673 papers whose bibliographies we read[Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)](https://aclanthology.org/volumes/L10-1/)[licence](https://creativecommons.org/licenses/by-nc/4.0/)

- [Publisher's page](https://aclanthology.org/L10-1248/)
- [PDF](https://aclanthology.org/L10-1248.pdf)
- [DOI 10.63317/36n37t2792qj](https://doi.org/10.63317/36n37t2792qj)
- [OpenAlex](https://openalex.org/W7688137)

**Authors (7)**

1. [Paul Felt](https://openalex.org/A5060115458) [Semantic Scholar](https://www.semanticscholar.org/author/1873296) · h 7 · 106 citations · 20 works
2. [Owen Merkling](https://openalex.org/A5044980625) [Semantic Scholar](https://www.semanticscholar.org/author/3216154) · h 2 · 16 citations · 2 works
3. [Marc Carmen](https://openalex.org/A5036705308) [Semantic Scholar](https://www.semanticscholar.org/author/153368281) · h 4 · 125 citations · 12 works
4. [Eric Ringger](https://openalex.org/A5090791372) [Semantic Scholar](https://www.semanticscholar.org/author/1772625) · h 21 · 1822 citations · 103 works
5. [Warren Lemmon](https://openalex.org/A5053072814) [Semantic Scholar](https://www.semanticscholar.org/author/47405035) · h 2 · 18 citations · 2 works
6. [Kevin Seppi](https://openalex.org/A5029074659) [ORCID](https://orcid.org/0000-0002-4932-2700) · [Semantic Scholar](https://www.semanticscholar.org/author/31819161) · h 23 · 1923 citations · 137 works
7. [Robbie Haertel](https://openalex.org/A5017536910) [Semantic Scholar](https://www.semanticscholar.org/author/1963625) · h 9 · 275 citations · 31 works

Citations and author metrics from openalex (2026-10-19); authors with the same name are not always told apart.

## What it is about

CCASH is a web application framework for corpus annotation, built so that cost-efficient methods such as active learning and automatic pre-labelling can first be tried in user studies and then run on large projects. The browser client is written in GWT, and the server keeps instances and annotations in a database through Hibernate. The instance provider, which selects and pre-labels items, runs as a separate service over XML-RPC and can be written in any language. Every annotation is logged as timed events (requested, presented, completed, paused, resumed), so cost can be measured per item. Two tasks are built in, English POS tagging and NER. The case study describes a user study of 33 graduate students with tag-dictionary coverage levels; the paper describes the design of that study rather than giving its results.

## Ideas

- I1. Log a timed event for each tree when it is shown, edited and saved, and derive time per token. This measures what it costs to correct the Hebrew trees from `tools/hbo` drafts, and it is the first measure to put on the correction workflow for `treebanks/hbo/ptnk`.
- I2. A balanced design for a pre-labelling study: each annotator sees each of six suggestion levels exactly three times, on the same 18 sentences in a fixed order, with sentence lengths matched across levels. Use this to compare pointed and unpointed pre-labels in `tools/hbo`.
- I3. Keep the editor and the parser apart, with the parser behind a plain request interface, so that a new segmenter or parser can be swapped in without changing the editor. This is an architecture note for `tools/hbo`, not a rule.

## For Mova

I1 goes into the correction workflow for `treebanks/hbo/ptnk`. I2 goes into the evaluation plan for `tools/hbo`. I3 is a design note for `tools/hbo`. The paper's Java and GWT code is not used.

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.63317/36n37t2792qj](https://doi.org/10.63317/36n37t2792qj)

```
@inproceedings{felt2010ccash,
  title = {{CCASH: A Web Application Framework for Efficient, Distributed Language Resource Development}},
  author = {Felt, Paul and Merkling, Owen and Carmen, Marc and Ringger, Eric and Lemmon, Warren and Seppi, Kevin and Haertel, Robbie},
  booktitle = {Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)},
  year = {2010},
  doi = {10.63317/36n37t2792qj},
  url = {https://doi.org/10.63317/36n37t2792qj}
}
```

- [← previous Machine Translation between English and Syriac (East Syriac Dialect) using Statistical Machine Learning](https://md.mova.elamur.ai/free/en/docs/papers/langstream-2026-sliwa-machine-translation-between-english-syriac/)
- [next → First Results in a Study Evaluating Pre-annotation and Correction Propagation for Machine-Assisted Syriac Morphological Analysis](https://md.mova.elamur.ai/free/en/docs/papers/lingbulk-2012-felt-first-results-study-evaluating-pre/)
- [↑ back to Aramaic](https://md.mova.elamur.ai/free/en/docs/papers/arc/)

## References

18 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Carmen, M., Felt, P., Haertel, R., Lonsdale, D., McClanahan, P., Merkling, O., Ringger, E., Seppi, K. (2010). Tag Dictionaries Accelerate Manual Annotation. In Proceedings of LREC 2010. Malta. [↗](https://doi.org/10.63317/2wvufaog66zv)
2. Chiou, F.-D., Chiang, D., & Palmer, M. (2001). Facilitating Treebank Annotation with a Statistical Parser. In Proceedings of the Human Language Technology (HLT) Conference. San Diego: ACL. [find](https://scholar.google.com/scholar?q=Chiou%2C+F%2E%2DD%2E%2C+Chiang%2C+D%2E%2C+%26+Palmer%2C+M%2E+%282001%29%2E+Facilitating+Treebank+Annotation+with+a+Statistical+Parser%2E+In+Proceedings+of+the+Human+Language+Technology+%28HLT%29+Conference%2E+San+Diego%3A+ACL%2E)
3. Culotta, A. & McCallum, A. (2005). Reducing labeling effort for stuctured prediction tasks. In The Twentieth National Conference on Artificial Intelligence (AAAI). Pittsburgh, PA: pp. 746-751. [find](https://scholar.google.com/scholar?q=Culotta%2C+A%2E+%26+McCallum%2C+A%2E+%282005%29%2E+Reducing+labeling+effort+for+stuctured+prediction+tasks%2E+In+The+Twentieth+National+Conference+on+Artificial+Intelligence+%28AAAI%29%2E+Pittsburgh%2C+PA%3A+pp%2E+746%2D751%2E)
4. Cunningham, H., Maynard, D., Kalina, B., & Tablan, V. (2002). GATE: A Framework and Graphical Development Environment for Robust NLP Tools and Applications. In Proceedings of the 40th Anniversary Meeting of the ACL. Philadelphia. [find](https://scholar.google.com/scholar?q=Cunningham%2C+H%2E%2C+Maynard%2C+D%2E%2C+Kalina%2C+B%2E%2C+%26+Tablan%2C+V%2E+%282002%29%2E+GATE%3A+A+Framework+and+Graphical+Development+Environment+for+Robust+NLP+Tools+and+Applications%2E+In+Proceedings+of+the+40th+Anniversary+Meet)
5. Ganchev, K., Pereira, F., & Mandel, M. (2007). Semi-automated Named Entity Annotation. In Proceedings of the Linguistic Annotation Workshop. Prague: ACL, pp. 53-56. [↗](https://doi.org/10.3115/1642059.1642068)
6. Haertel, R., Ringger, E., Seppi, K., Carroll, J., & McClanahan, P. (2008a). Assessing the Costs of Sampling Methods in Active Learning for Annotation. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies. Columbus, Ohio: ACL Short Papers, pp. 65-68. [↗](https://doi.org/10.3115/1557690.1557708)
7. Haertel, R. A., Seppi, K. D., Ringger, E. K., & Carroll, J. (2008b). Return on Investment for Active Learning. In Proceedings of the NIPS Workshop on Cost-Sensitive Learning. ACL Press. [find](https://scholar.google.com/scholar?q=Haertel%2C+R%2E+A%2E%2C+Seppi%2C+K%2E+D%2E%2C+Ringger%2C+E%2E+K%2E%2C+%26+Carroll%2C+J%2E+%282008b%29%2E+Return+on+Investment+for+Active+Learning%2E+In+Proceedings+of+the+NIPS+Workshop+on+Cost%2DSensitive+Learning%2E+ACL+Press%2E)
8. Lowe, J., Jacobson, M., & Michailovsky, B. (2004). Interlinear Text Editor Demonstration and Project Archivage Progress Report. In 4th EMELD Work-shop on Linguistic Databases and Best Practice. Detroit. [find](https://scholar.google.com/scholar?q=Lowe%2C+J%2E%2C+Jacobson%2C+M%2E%2C+%26+Michailovsky%2C+B%2E+%282004%29%2E+Interlinear+Text+Editor+Demonstration+and+Project+Archivage+Progress+Report%2E+In+4th+EMELD+Work%2Dshop+on+Linguistic+Databases+and+Best+Practice%2E+Detroi)
9. Marcus, M., Santorini, B., & Marcinkiewicz, M. (1994). Building a Large Annotated Corpus of English: The Penn Treebank. Computational Linguistics 19(2), pp. 313-330. [↗](https://doi.org/10.21236/ada273556) [our card](https://md.mova.elamur.ai/free/en/docs/papers/bank-1993-marcus-building-large-annotated-corpus-english/)
10. Morton, T., & Lacivita, J. (2003). Word-Freak: An Open Tool for Linguistic Annotation. In Proceedings of HLT-NAACL, pp. 17-18. [find](https://scholar.google.com/scholar?q=Morton%2C+T%2E%2C+%26+Lacivita%2C+J%2E+%282003%29%2E+Word%2DFreak%3A+An+Open+Tool+for+Linguistic+Annotation%2E+In+Proceedings+of+HLT%2DNAACL%2C+pp%2E+17%2D18%2E)
11. Müller, C., & Strube, M. (2006). Multi-Level Annotation of Linguistic Data with MMAX2. In S. Braun, K. Kohn, & J. Mukherjee (Eds.), Corpus Technology and Language Pedagogy. New Resources, New Tools, New Methods. English Corpus Linguistics Vol. 3. Frankfurt: Peter Lang, pp. 197-214. [find](https://scholar.google.com/scholar?q=M%C3%BCller%2C+C%2E%2C+%26+Strube%2C+M%2E+%282006%29%2E+Multi%2DLevel+Annotation+of+Linguistic+Data+with+MMAX2%2E+In+S%2E+Braun%2C+K%2E+Kohn%2C+%26+J%2E+Mukherjee+%28Eds%2E%29%2C+Corpus+Technology+and+Language+Pedagogy%2E+New+Resources%2C+New+Tools%2C+N)
12. Ngai, G., & Yarowsky, D. (2000). Rule Writing or Annotation: Cost-efficient Resource Usage for Base Noun Phrase Chunking. In Proceedings of ACL, pp. 117-125. [↗](https://doi.org/10.3115/1075218.1075234)
13. Ogren, P. V. (2006). Knowtator: A Protégé plug-in for Annotated Corpus Construction. In Proceedings of the 2006 Conference of the NAACL-HLT. New York: Companion Volume, Demonstrations, pp. 273-275. [↗](https://doi.org/10.3115/1225785.1225791)
14. Ringger, E., McClanahan, P., Haertel, R., Busby, G., Carmen, M., Carroll, J., et al. (2007). Active Learning for Part-of-speech Tagging: Accelerating Corpus Annotation. In Proceedings of the Linguistic Annotation Workshop at ACL, pp. 101-108. [↗](https://doi.org/10.3115/1642059.1642075)
15. Ringger, E., Carmen, M., Haertel, R., Seppi, K., Lonsdale D., McClanahan P., Carroll, J., & Ellison, N.. (2008). Assessing the Costs of Machine-assisted Corpus Annotation through a User Study. In Proceedings of LREC 2008. Morocco. [↗](https://doi.org/10.63317/3afovxbya4dv)
16. Settles, B., Craven, M., & Friedland L. 2008. Active Learning with Real Annotation Costs. In Proceedings of the NIPS Workshop on Cost-Sensitive Learning, pp. 1069-1078. [find](https://scholar.google.com/scholar?q=Settles%2C+B%2E%2C+Craven%2C+M%2E%2C+%26+Friedland+L%2E+2008%2E+Active+Learning+with+Real+Annotation+Costs%2E+In+Proceedings+of+the+NIPS+Workshop+on+Cost%2DSensitive+Learning%2C+pp%2E+1069%2D1078%2E)
17. Settles, B. (2009). Active Learning Literature Survey. University of Wisconsin-Madison. Computer Sciences Technical Report 1648. [find](https://scholar.google.com/scholar?q=Settles%2C+B%2E+%282009%29%2E+Active+Learning+Literature+Survey%2E+University+of+Wisconsin%2DMadison%2E+Computer+Sciences+Technical+Report+1648%2E)
18. Tomanek, K., Wermter, J., & Hahn, U. (2007). Efficient Annotation with the Jena ANnotation Environment (JANE). In Proceedings of the ACL 2007 Linguistic Annotation Workshop–A Merger of NLPXML 2007 and FLAC. Prague. [↗](https://doi.org/10.3115/1642059.1642061)