[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/aclbulk-2010-litvak-multi-lingual-summarization-comparative-analysis/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [Modern Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/he/) / Towards multi-lingual summarization: A comparative analysis of sentence extraction methods on English and Hebrew corpora

# Towards multi-lingual summarization: A comparative analysis of sentence extraction methods on English and Hebrew corpora

**Authors:** [Marina Litvak](https://md.mova.elamur.ai/free/en/docs/papers/authors/marina-litvak/); [Mark Last](https://md.mova.elamur.ai/free/en/docs/papers/authors/mark-last/); Slava Kisilevich; Daniel Keim; Hagay Lipman; Assaf Ben Gur · **Year:** 2010 · **Venue:** Proceedings of the 4th Workshop on Cross Lingual Information Access **Link:** [aclanthology.org/W10-4010](https://aclanthology.org/W10-4010/) · library `aclbulk-2010-litvak-multi-lingual-summarization-comparative-analysis` · **Licence:** [CC BY-NC-SA 3.0](https://creativecommons.org/licenses/by-nc-sa/3.0/) (ACL Anthology, before 2016) — summary only

[**4** citations](https://openalex.org/works?filter=cites:W598553093) openalex-snapshot · 2026-10-14[**1** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/aclbulk-2010-litvak-multi-lingual-summarization-comparative-analysis/) of the 140673 papers whose bibliographies we read[Proceedings of the 4th Workshop on Cross Lingual Information Access](https://aclanthology.org/volumes/W10-40/)[licence](https://creativecommons.org/licenses/by-nc-sa/3.0/)

- [Publisher's page](https://aclanthology.org/W10-4010/)
- [PDF](https://aclanthology.org/W10-4010.pdf)
- [OpenAlex](https://openalex.org/W598553093)

**Authors (6)**

1. [Marina Litvak](https://openalex.org/A5031572202) [ORCID](https://orcid.org/0000-0003-3044-3681) · [Semantic Scholar](https://www.semanticscholar.org/author/143752322) · h 15 · 1030 citations · 114 works
2. Mark Last [Semantic Scholar](https://www.semanticscholar.org/author/3045152)
3. [Slava Kisilevich](https://openalex.org/A5055364640) [Semantic Scholar](https://www.semanticscholar.org/author/1910307) · h 16 · 1240 citations · 49 works
4. [Daniel Keim](https://openalex.org/A5073919282) [ORCID](https://orcid.org/0000-0001-7966-9740) · [Semantic Scholar](https://www.semanticscholar.org/author/9106757) · h 77 · 30490 citations · 854 works
5. [Hagay Lipman](https://openalex.org/A5036148692) [Semantic Scholar](https://www.semanticscholar.org/author/40526008) · h 1 · 4 citations · 1 works
6. [Assaf Ben Gur](https://openalex.org/A5015932868) h 1 · 4 citations · 1 works

Citations and author metrics from openalex-snapshot (2026-10-14); authors with the same name are not always told apart.

## What it is about

The paper compares 31 language-independent methods for scoring sentences for extractive single-document summaries: 15 published ones and 16 graph and vector methods introduced by the same group earlier. The methods use no morphological analysis, so they can be applied to any language. They are tested on English news (DUC 2002, 533 articles) and on a new Hebrew set of 50 Haaretz articles that 70 students summarised by picking sentences. Scores are ROUGE-1 recall. The rankings in the two languages correlate at 0.775 (Pearson), and six of the top ten bilingual methods are the new ones (for example document-coverage with Jaccard or cosine, and graph variants of coverage and keyword scoring). Because Hebrew affixes make word counts unstable, the character-length measure is recommended over the word-length one for multilingual use.

## Ideas

- I1. A cross-language check for any Mova summariser or retrieval scorer that is meant to work on Hebrew as well as English: run the same ranking on both languages and report the rank correlation, as the paper does. This is a measurement, not a model, and it fits the English-first RAG work in `world` and `prag`.
- I2. Use character length rather than word length as the length feature for Hebrew sentences, because clitics (prefixed particles and pronominal suffixes) change the word count of the same content. A small test on `treebanks/he/` sentences would show whether word counts are unstable there.
- I3. The Hebrew experiment design (each article summarised by several students, each student given a fixed sample, a Friedman test and then Bonferroni-Dunn pairwise comparisons) is a template for a small, clean evaluation set when Mova needs its own gold summaries.

## For Mova

I1 is a measurement for the summary and retrieval tests of `world` and `prag`. I2 goes into [2. Script and spelling](https://md.mova.elamur.ai/free/en/docs/languages/he/02-script-and-spelling/) as a note on tokenisation and into any length feature in `tools/langseg` output. I3 is a test-design note for Mova's own held-out sets, not a data source: the corpora here are non-commercial and only for tests.

## Cite

The paper itself, not this card. BibTeX from the library's record of the work.

```
@inproceedings{litvak2010multi,
  title = {{Towards multi-lingual summarization: A comparative analysis of sentence extraction methods on English and Hebrew corpora}},
  author = {Litvak, Marina and Last, Mark and Kisilevich, Slava and Keim, Daniel and Lipman, Hagay and Gur, Assaf Ben},
  booktitle = {Proceedings of the 4th Workshop on Cross Lingual Information Access},
  year = {2010},
  url = {https://aclanthology.org/W10-4010/}
}
```

- [← previous Easy-First Dependency Parsing of Modern Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2010-goldberg-easy-first-dependency-parsing-modern/)
- [next → Modeling Morphosyntactic Agreement in Constituency-Based Parsing of Modern Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/aclbulk-2010-tsarfaty-modeling-morphosyntactic-agreement-constituency-based/)
- [↑ back to Modern Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/he/)

## References

26 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Adler, M. (2009). Hebrew morphological disambiguation: An unsupervised stochastic word-based approach. Dissertation. [http://www.cs.bgu.ac.il/](http://www.cs.bgu.ac.il/) adlerm/dat/thesis.pdf. [↗](http://www.cs.bgu.ac.il/)
2. Baxendale, P. (1958). Machine-made index for technical literature-an experiment. IBM Journal of Research and Development, 2, 354–361. [↗](https://doi.org/10.1147/rd.24.0354)
3. Brin, S., & Page, L. (1998). The anatomy of a large-scale hypertextual web search engine. Computer networks and ISDN systems, 30, 107–117. [↗](https://doi.org/10.1016/s0169-7552(98)00110-x)
4. Dang, H. T. (2006). Overview of DUC 2006. Proceedings of the Document Understanding Conference. [find](https://scholar.google.com/scholar?q=Dang%2C+H%2E+T%2E+%282006%29%2E+Overview+of+DUC+2006%2E+Proceedings+of+the+Document+Understanding+Conference%2E)
5. Demsar, J. (2006). Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7, 1–30. [find](https://scholar.google.com/scholar?q=Demsar%2C+J%2E+%282006%29%2E+Statistical+comparisons+of+classifiers+over+multiple+data+sets%2E+Journal+of+Machine+Learning+Research%2C+7%2C+1%E2%80%9330%2E)
6. DUC (2002). Document understanding conference. [http://duc.nist.gov](http://duc.nist.gov). [↗](http://duc.nist.gov)
7. Edmundson, H. P. (1969). New methods in automatic extracting. J. ACM, 16. [↗](https://doi.org/10.1145/321510.321519)
8. Erkan, G., & Radev, D. R. (2004). LexRank: Graph-based lexical centrality as salience in text summarization. Journal of Artificial Intelligence Research, 22, 457–479. Our experiments have shown that syntactic filters, which select only lexical units of a certain part of speech, do not significantly improve the performance of the evaluated bilingual scoring methods. [↗](https://doi.org/10.1613/jair.1523)
9. Gong, Y., & Liu, X. (2001). Generic text summarization using relevance measure and latent semantic analysis. Proceedings of the 24th ACM SIGIR conference on Research and development in information retrieval (pp. 19–25). [find](https://scholar.google.com/scholar?q=Gong%2C+Y%2E%2C+%26+Liu%2C+X%2E+%282001%29%2E+Generic+text+summarization+using+relevance+measure+and+latent+semantic+analysis%2E+Proceedings+of+the+24th+ACM+SIGIR+conference+on+Research+and+development+in+information+ret)
10. Kallel, F. J., Jaoua, M., Hadrich, L. B., & Hamadou, A. B. (2004). Summarization at LARIS laboratory. Proceedings of the Document Understanding Conference. [find](https://scholar.google.com/scholar?q=Kallel%2C+F%2E+J%2E%2C+Jaoua%2C+M%2E%2C+Hadrich%2C+L%2E+B%2E%2C+%26+Hamadou%2C+A%2E+B%2E+%282004%29%2E+Summarization+at+LARIS+laboratory%2E+Proceedings+of+the+Document+Understanding+Conference%2E)
11. Kupiec, J., Pedersen, J., & Chen, F. (1995). A trainable document summarizer. Proceedings of the 18th annual international ACM SIGIR conference (pp. 68–73). [find](https://scholar.google.com/scholar?q=Kupiec%2C+J%2E%2C+Pedersen%2C+J%2E%2C+%26+Chen%2C+F%2E+%281995%29%2E+A+trainable+document+summarizer%2E+Proceedings+of+the+18th+annual+international+ACM+SIGIR+conference+%28pp%2E+68%E2%80%9373%29%2E)
12. Lin, C.-Y. (2004). ROUGE: A package for automatic evaluation of summaries. Proceedings of the ACL’04 Workshop: Text Summarization Branches Out (pp. 74–81). [find](https://scholar.google.com/scholar?q=Lin%2C+C%2E%2DY%2E+%282004%29%2E+ROUGE%3A+A+package+for+automatic+evaluation+of+summaries%2E+Proceedings+of+the+ACL%E2%80%9904+Workshop%3A+Text+Summarization+Branches+Out+%28pp%2E+74%E2%80%9381%29%2E)
13. Litvak, M., Last, M., & Friedman, M. (2010). A new approach to improving multilingual summarization using a genetic algorithm. Proceedings of the Association for Computational Linguistics (ACL) 2010. Uppsala, Sweden. [find](https://scholar.google.com/scholar?q=Litvak%2C+M%2E%2C+Last%2C+M%2E%2C+%26+Friedman%2C+M%2E+%282010%29%2E+A+new+approach+to+improving+multilingual+summarization+using+a+genetic+algorithm%2E+Proceedings+of+the+Association+for+Computational+Linguistics+%28ACL%29+2010%2E+)
14. Luhn, H. P. (1958). The automatic creation of literature abstracts. IBM Journal of Research and Development, 2, 159–165. [↗](https://doi.org/10.1147/rd.22.0159)
15. Mani, I., & Maybury, M. (1999). Advances in automatic text summarization. [find](https://scholar.google.com/scholar?q=Mani%2C+I%2E%2C+%26+Maybury%2C+M%2E+%281999%29%2E+Advances+in+automatic+text+summarization%2E)
16. Mihalcea, R. (2005). Language independent extractive summarization. AAAI’05: Proceedings of the 20th national conference on Artificial intelligence (pp. 1688–1689). [find](https://scholar.google.com/scholar?q=Mihalcea%2C+R%2E+%282005%29%2E+Language+independent+extractive+summarization%2E+AAAI%E2%80%9905%3A+Proceedings+of+the+20th+national+conference+on+Artificial+intelligence+%28pp%2E+1688%E2%80%931689%29%2E)
17. Nastase, V., & Szpakowicz, S. (2006). A study of two graph algorithms in topic-driven summarization. Proceedings of the Workshop on Graph-based Algorithms for Natural Language. [find](https://scholar.google.com/scholar?q=Nastase%2C+V%2E%2C+%26+Szpakowicz%2C+S%2E+%282006%29%2E+A+study+of+two+graph+algorithms+in+topic%2Ddriven+summarization%2E+Proceedings+of+the+Workshop+on+Graph%2Dbased+Algorithms+for+Natural+Language%2E)
18. Neto, J., Santos, A., Kaestner, C., & Freitas, A. (2000). Generating text summaries through the relative importance of topics. Lecture Notes in Computer Science, 300–309. [find](https://scholar.google.com/scholar?q=Neto%2C+J%2E%2C+Santos%2C+A%2E%2C+Kaestner%2C+C%2E%2C+%26+Freitas%2C+A%2E+%282000%29%2E+Generating+text+summaries+through+the+relative+importance+of+topics%2E+Lecture+Notes+in+Computer+Science%2C+300%E2%80%93309%2E)
19. Radev, D., Blair-Goldensohn, S., & Zhang, Z. (2001). Experiments in single and multidocument summarization using MEAD. First Document Understanding Conference. [find](https://scholar.google.com/scholar?q=Radev%2C+D%2E%2C+Blair%2DGoldensohn%2C+S%2E%2C+%26+Zhang%2C+Z%2E+%282001%29%2E+Experiments+in+single+and+multidocument+summarization+using+MEAD%2E+First+Document+Understanding+Conference%2E)
20. Saggion, H., Bontcheva, K., & Cunningham, H. (2003). Robust generic and query-based summarisation. EACL ’03: Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Saggion%2C+H%2E%2C+Bontcheva%2C+K%2E%2C+%26+Cunningham%2C+H%2E+%282003%29%2E+Robust+generic+and+query%2Dbased+summarisation%2E+EACL+%E2%80%9903%3A+Proceedings+of+the+tenth+conference+on+European+chapter+of+the+Association+for+Computationa)
21. Salton, G., Singhal, A., Mitra, M., & Buckley, C. (1997). Automatic text structuring and summarization. Information Processing and Management, 33, 193–207. [↗](https://doi.org/10.1016/s0306-4573(96)00062-3)
22. Satoshi, C. N., Satoshi, S., Murata, M., Uchimoto, K., Utiyama, M., & Isahara, H. (2001). Sentence extraction system assembling multiple evidence. Proceedings of 2nd NTCIR Workshop (pp. 319–324). [find](https://scholar.google.com/scholar?q=Satoshi%2C+C%2E+N%2E%2C+Satoshi%2C+S%2E%2C+Murata%2C+M%2E%2C+Uchimoto%2C+K%2E%2C+Utiyama%2C+M%2E%2C+%26+Isahara%2C+H%2E+%282001%29%2E+Sentence+extraction+system+assembling+multiple+evidence%2E+Proceedings+of+2nd+NTCIR+Workshop+%28pp%2E+319%E2%80%93324%29%2E)
23. Schenker, A., Bunke, H., Last, M., & Kandel, A. (2004). Classification of web documents using graph matching. International Journal of Pattern Recognition and Artificial Intelligence, 18, 475–496. [↗](https://doi.org/10.1142/s0218001404003241)
24. Steinberger, J., & Jezek, K. (2004). Text summarization and singular value decomposition. Lecture Notes in Computer Science, 245–254. [find](https://scholar.google.com/scholar?q=Steinberger%2C+J%2E%2C+%26+Jezek%2C+K%2E+%282004%29%2E+Text+summarization+and+singular+value+decomposition%2E+Lecture+Notes+in+Computer+Science%2C+245%E2%80%93254%2E)
25. Vanderwende, L., Suzuki, H., Brockett, C., & Nenkova, A. (2007). Beyond SumBasic: Task-focused summarization with sentence simplification and lexical expansion. Information processing and management, 43, 1606–1618. [↗](https://doi.org/10.1016/j.ipm.2007.01.023)
26. Wong, K., Wu, M., & Li, W. (2008). Extractive summarization using supervised and semi-supervised learning. Proceedings of the 22nd International Conference on Computational Linguistics (pp. 985–992). [find](https://scholar.google.com/scholar?q=Wong%2C+K%2E%2C+Wu%2C+M%2E%2C+%26+Li%2C+W%2E+%282008%29%2E+Extractive+summarization+using+supervised+and+semi%2Dsupervised+learning%2E+Proceedings+of+the+22nd+International+Conference+on+Computational+Linguistics+%28pp%2E+985%E2%80%93992%29%2E)