[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/cited-by/isabelle-2017-challenge-set-approach-evaluating-machine/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [cited-by](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/) / Who cites «A Challenge Set Approach to Evaluating Machine Translation»

# Who cites «A Challenge Set Approach to Evaluating Machine Translation»

111 papers of our library cite it, newest first: found in their bibliographies, which we read ourselves. [← the card](https://md.mova.elamur.ai/free/en/docs/papers/isabelle-2017-challenge-set-approach-evaluating-machine/)

Pages: **1** · [2](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/isabelle-2017-challenge-set-approach-evaluating-machine-2/)

1. [A Fine-Grained Linguistic Evaluation of Low-Resource Luxembourgish–English MT](https://aclanthology.org/2026.loresmt-1.12/) *— 2026 · Proceedings for the Ninth Workshop on Technologies for Machine Translation of Low Resource Languages (LoResMT 2026)*
2. [Benchmarking Multilingual LLM Translation Accuracy for Fuzhounese](https://aclanthology.org/2026.sigul-1.20/) *— 2026 · Proceedings of the SIGUL 2026 Joint Workshop with ELE, EURALI, and DCLRL: Towards Inclusivity and Equality: Language Resources and Technologies for Under-Resourced and Endangered Languages*
3. [Do Evaluation Metrics Detect Errors in Classical Chinese to English Translations?](https://arxiv.org/abs/2608.08283) *— 2026 · arXiv:2608.08283*
4. [Generating Difficult-to-Translate Texts](https://aclanthology.org/2026.mme-main.14/) *— 2026 · Proceedings of the First Workshop on Multilingual Multicultural Evaluation*
5. [Round-Trip Translation Reveals What Frontier Multilingual Benchmarks Miss](https://arxiv.org/abs/2604.12911) *— 2026 · arXiv:2604.12911*
6. [The Validity of Coreference-based Evaluations of Natural Language Understanding](https://arxiv.org/abs/2602.16200) *— 2026 · arXiv:2602.16200*
7. [A Large-Scale Benchmark for Vietnamese Sentence Paraphrases](https://aclanthology.org/2025.findings-naacl.59/) *— 2025 · Findings of the Association for Computational Linguistics: NAACL 2025*
8. [A Testset for Context-Aware LLM Translation in Korean-to-English Discourse Level Translation](https://aclanthology.org/2025.coling-main.110/) *— 2025 · Proceedings of the 31st International Conference on Computational Linguistics*
9. [GENDEROUS: Machine Translation and Cross-Linguistic Evaluation of a Gender-Ambiguous Dataset](https://aclanthology.org/2025.gebnlp-1.27/) *— 2025 · Proceedings of the 6th Workshop on Gender Bias in Natural Language Processing (GeBNLP)*
10. [MSLC25: Metric Performance on Low-Quality Machine Translation, Empty Strings, and Language Variants](https://aclanthology.org/2025.wmt-1.69/) *— 2025 · Proceedings of the Tenth Conference on Machine Translation*
11. [Machine Translation Meta Evaluation through Translation Accuracy Challenge Sets](https://aclanthology.org/2025.cl-1.4/) *— 2025 · Computational Linguistics*
12. [Proverbs Run in Pairs: Evaluating Proverb Translation Capability of Large Language Model](https://aclanthology.org/2025.findings-acl.83/) *— 2025 · Findings of the Association for Computational Linguistics: ACL 2025*
13. [Using LLMs to Advance Idiom Corpus Construction](https://aclanthology.org/2025.mwe-1.4/) *— 2025 · Proceedings of the 21st Workshop on Multiword Expressions (MWE 2025)*
14. [A Paradigm Shift: The Future of Machine Translation Lies with Large Language Models](https://aclanthology.org/2024.lrec-main.120/) *— 2024 · Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)*
15. [Context-Aware Neural Machine Translation Models Analysis And Evaluation Through Attention](https://aclanthology.org/2023.tal-3.4/) *— 2024*
16. [GPT-HateCheck: Can LLMs Write Better Functional Tests for Hate Speech Detection?](https://aclanthology.org/2024.lrec-main.694/) *— 2024 · Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)*
17. [Quality and Quantity of Machine Translation References for Automatic Metrics](https://aclanthology.org/2024.humeval-1.1/) *— 2024 · Proceedings of the Fourth Workshop on Human Evaluation of NLP Systems (HumEval) @ LREC-COLING 2024*
18. [Solving the Challenge Set without Solving the Task: On Winograd Schemas as a Test of Pronominal Coreference Resolution](https://aclanthology.org/2024.conll-1.38/) *— 2024 · Proceedings of the 28th Conference on Computational Natural Language Learning*
19. [Évaluation de la Similarité Textuelle : Entre Sémantique et Surface dans les Représentations Neuronales](https://aclanthology.org/2024.jeptalnrecital-taln.6/) *— 2024 · Actes de la 31ème Conférence sur le Traitement Automatique des Langues Naturelles, volume 1 : articles longs et prises de position*
20. [A Multilingual Evaluation of NER Robustness to Adversarial Inputs](https://aclanthology.org/2023.repl4nlp-1.4/) *— 2023 · Proceedings of the 8th Workshop on Representation Learning for NLP (RepL4NLP 2023)*
21. [ACES: Translation Accuracy Challenge Sets at WMT 2023](https://aclanthology.org/2023.wmt-1.57/) *— 2023 · Proceedings of the Eighth Conference on Machine Translation*
22. [Assessing the Capacity of Transformer to Abstract Syntactic Representations: A Contrastive Analysis Based on Long-distance Agreement](https://aclanthology.org/2023.tacl-1.2/) *— 2023 · Transactions of the Association for Computational Linguistics*
23. [Discourse Centric Evaluation of Machine Translation with a Densely Annotated Parallel Corpus](https://arxiv.org/abs/2305.11142) *— 2023 · arXiv:2305.11142 · ACL 2023*
24. [Discourse-Centric Evaluation of Document-level Machine Translation with a New Densely Annotated Parallel Corpus of Novels](https://aclanthology.org/2023.acl-long.435/) *— 2023 · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*
25. [Fighting Bias With Bias: Promoting Model Robustness by Amplifying Dataset Biases](https://aclanthology.org/2023.findings-acl.833/) *— 2023 · Findings of the Association for Computational Linguistics: ACL 2023*
26. [Linguistically Motivated Evaluation of the 2023 State-of-the-art Machine Translation: Can ChatGPT Outperform NMT?](https://aclanthology.org/2023.wmt-1.23/) *— 2023 · Proceedings of the Eighth Conference on Machine Translation*
27. [A Linguistically Motivated Test Suite to Semi-Automatically Evaluate German–English Machine Translation Output](https://aclanthology.org/2022.lrec-1.99/) *— 2022 · Proceedings of the Thirteenth Language Resources and Evaluation Conference*
28. [ACES: Translation Accuracy Challenge Sets for Evaluating Machine Translation Metrics](https://md.mova.elamur.ai/free/en/docs/papers/amrhein-2022-aces-translation-accuracy-challenge-sets/) *— our card · 2022 · Proceedings of the Seventh Conference on Machine Translation (WMT)*
29. [BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation](https://aclanthology.org/2022.naacl-main.111/) *— 2022 · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies*
30. [Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translation](https://aclanthology.org/2022.acl-long.252/) *— 2022 · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*
31. [Food for Thought: How can we exploit contextual embeddings in the translation of idiomatic expressions?](https://aclanthology.org/2022.flp-1.14/) *— 2022 · Proceedings of the 3rd Workshop on Figurative Language Processing (FLP)*
32. [Investigating Failures of Automatic Translation in the Case of Unambiguous Gender](https://aclanthology.org/2022.acl-long.243/) *— 2022 · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*
33. [Linguistically Motivated Evaluation of the 2022 State-of-the-art Machine Translation Systems for Three Language Directions](https://md.mova.elamur.ai/free/en/docs/papers/macketanz-2022-linguistically-motivated-evaluation-2022-state/) *— our card · 2022 · Proceedings of the Seventh Conference on Machine Translation (WMT)*
34. [Linguistically-Motivated Yorùbá-English Machine Translation](https://aclanthology.org/2022.coling-1.449/) *— 2022 · Proceedings of the 29th International Conference on Computational Linguistics*
35. [PreQuEL: Quality Estimation of Machine Translation Outputs in Advance](https://aclanthology.org/2022.emnlp-main.767/) *— 2022 · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing*
36. [Test Suite Evaluation: Morphological Challenges and Pronoun Translation](https://aclanthology.org/2022.wmt-1.42/) *— 2022 · Proceedings of the Seventh Conference on Machine Translation (WMT)*
37. [Toward a Test Set of Dislocations in Persian for Neural Machine Translation](https://aclanthology.org/2022.nsurl-1.3/) *— 2022 · Proceedings of the Third International Workshop on NLP Solutions for Under Resourced Languages (NSURL 2022) co-located with ICNLSP 2022*
38. [Biais de genre dans un système de traduction automatiqueneuronale : une étude préliminaire (Gender Bias in Neural Translation : a preliminary study )](https://aclanthology.org/2021.jeptalnrecital-taln.2/) *— 2021 · Actes de la 28e Conférence sur le Traitement Automatique des Langues Naturelles. Volume 1 : conférence principale*
39. [CausaLM: Causal Model Explanation Through Counterfactual Language Models](https://aclanthology.org/2021.cl-2.13/) *— 2021 · Computational Linguistics*
40. [Comparing Knowledge-Intensive and Data-Intensive Models for English Resource Semantic Parsing](https://aclanthology.org/2021.cl-1.3/) *— 2021 · Computational Linguistics*
41. [Contextualized Embeddings Encode Monolingual and Cross-lingual Knowledge of Idiomaticity](https://aclanthology.org/2021.mwe-1.4/) *— 2021 · Proceedings of the 17th Workshop on Multiword Expressions (MWE 2021)*
42. [Does BERT Understand Idioms? A Probing-Based Empirical Study of BERT Encodings of Idioms](https://aclanthology.org/2021.ranlp-1.156/) *— 2021 · Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)*
43. [HateCheck: Functional Tests for Hate Speech Detection Models](https://aclanthology.org/2021.acl-long.4/) *— 2021 · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)*
44. [Linguistic Evaluation for the 2021 State-of-the-art Machine Translation Systems for German to English and English to German](https://aclanthology.org/2021.wmt-1.115/) *— 2021 · Proceedings of the Sixth Conference on Machine Translation*
45. [NoiseQA: Challenge Set Evaluation for User-Centric Question Answering](https://aclanthology.org/2021.eacl-main.259/) *— 2021 · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume*
46. [On nature and causes of observed MT errors](https://aclanthology.org/2021.mtsummit-research.14/) *— 2021 · Proceedings of Machine Translation Summit XVIII: Research Track*
47. [Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?](https://aclanthology.org/2021.eacl-main.295/) *— 2021 · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume*
48. [Sometimes We Want Ungrammatical Translations](https://aclanthology.org/2021.findings-emnlp.275/) *— 2021 · Findings of the Association for Computational Linguistics: EMNLP 2021*
49. [Tribrid: Stance Classification with Neural Inconsistency Detection](https://aclanthology.org/2021.emnlp-main.547/) *— 2021 · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing*
50. [An Error-based Investigation of Statistical and Neural Machine Translation Performance on Hindi-to-Tamil and English-to-Tamil](https://aclanthology.org/2020.wat-1.22/) *— 2020 · Proceedings of the 7th Workshop on Asian Translation*
51. [Cross-Linguistic Syntactic Evaluation of Word Prediction Models](https://aclanthology.org/2020.acl-main.490/) *— 2020 · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*
52. [Curious Case of Language Generation Evaluation Metrics: A Cautionary Tale](https://aclanthology.org/2020.coling-main.210/) *— 2020 · Proceedings of the 28th International Conference on Computational Linguistics*
53. [Disambiguation of Potentially Idiomatic Expressions with Contextual Embeddings](https://aclanthology.org/2020.mwe-1.11/) *— 2020 · Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons*
54. [Evaluating Models' Local Decision Boundaries via Contrast Sets](https://md.mova.elamur.ai/free/en/docs/papers/gardner-2020-evaluating-models-local-decision-boundaries/) *— our card · 2020 · Findings of the Association for Computational Linguistics: EMNLP 2020*
55. [Fine-grained linguistic evaluation for state-of-the-art Machine Translation](https://aclanthology.org/2020.wmt-1.38/) *— 2020 · Proceedings of the Fifth Conference on Machine Translation*
56. [No more rage against the machine: how the corpus-based identification of machine-translationese can lead to student empowerment](https://www.jostrans.org/article/view/7910) *— 2020 · The Journal of Specialised Translation (JoSTrans)*
57. [Online Versus Offline NMT Quality: An In-depth Analysis on English-German and German-English](https://aclanthology.org/2020.coling-main.443/) *— 2020 · Proceedings of the 28th International Conference on Computational Linguistics*
58. [PheMT: A Phenomenon-wise Dataset for Machine Translation Robustness on User-Generated Contents](https://aclanthology.org/2020.coling-main.521/) *— 2020 · Proceedings of the 28th International Conference on Computational Linguistics*
59. [Quantitative Analysis of Post-Editing Effort Indicators for NMT](https://aclanthology.org/2020.eamt-1.44/) *— 2020 · Proceedings of the 22nd Annual Conference of the European Association for Machine Translation*
60. [Stance Detection Benchmark: How Robust Is Your Stance Detection?](https://arxiv.org/abs/2001.01565) *— 2020 · arXiv:2001.01565*
61. [Analysis Methods in Neural Language Processing: A Survey](https://aclanthology.org/Q19-1004/) *— 2019 · Transactions of the Association for Computational Linguistics*
62. [Automatically Extracting Challenge Sets for Non-Local Phenomena in Neural Machine Translation](https://aclanthology.org/K19-1028/) *— 2019 · Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)*
63. [Context in Neural Machine Translation: A Review of Models and Evaluations](https://arxiv.org/abs/1901.09115) *— 2019 · arXiv:1901.09115*
64. [Does NMT make a difference when post-editing closely related languages? The case of Spanish-Catalan](https://aclanthology.org/W19-6708/) *— 2019 · Proceedings of Machine Translation Summit XVII: Translator, Project and User Tracks*
65. [Evaluating adversarial attacks against multiple fact verification systems](https://aclanthology.org/D19-1292/) *— 2019 · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)*
66. [Extract and Edit: An Alternative to Back-Translation for Unsupervised Neural Machine Translation](https://aclanthology.org/N19-1120/) *— 2019 · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)*
67. [Human Evaluation of Neural Machine Translation: The Case of Deep Learning](https://aclanthology.org/W19-8705/) *— 2019 · Proceedings of the Human-Informed Translation and Interpreting Technology Workshop (HiT-IT 2019)*
68. [Non-Parametric Adaptation for Neural Machine Translation](https://aclanthology.org/N19-1191/) *— 2019 · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)*
69. [Semantic Neural Machine Translation Using AMR](https://md.mova.elamur.ai/free/en/docs/papers/song-2019-semantic-neural-machine-translation-amr/) *— our card · 2019 · Transactions of the Association for Computational Linguistics*
70. [The MuCoW Test Suite at WMT 2019: Automatically Harvested Multilingual Contrastive Word Sense Disambiguation Test Sets for Machine Translation](https://aclanthology.org/W19-5354/) *— 2019 · Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1)*
71. [compare-mt: A Tool for Holistic Comparison of Language Generation Systems](https://aclanthology.org/N19-4007/) *— 2019 · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)*
72. [A fine-grained error analysis of NMT, SMT and RBMT output for English-to-Dutch](https://aclanthology.org/L18-1600/) *— 2018 · Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)*
73. [Analyzing Uncertainty in Neural Machine Translation](https://arxiv.org/abs/1803.00047) *— 2018 · arXiv:1803.00047*
74. [Automatic Evaluation of English-to-Korean and Korean-to-English Neural Machine Translation Systems by Linguistic Test Points](https://aclanthology.org/Y18-1013/) *— 2018 · Proceedings of the 32nd Pacific Asia Conference on Language, Information and Computation*
75. [Construction of a Multilingual Corpus Annotated with Translation Relations](https://aclanthology.org/W18-3814/) *— 2018 · Proceedings of the First Workshop on Linguistic Resources for Natural Language Processing*
76. [Evaluating Machine Translation Performance on Chinese Idioms with a Blacklist Method](https://aclanthology.org/L18-1005/) *— 2018 · Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)*
77. [Examining the Tip of the Iceberg: A Data Set for Idiom Translation](https://aclanthology.org/L18-1148/) *— 2018 · Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)*
78. [Exploiting Semantics in Neural Machine Translation with Graph Convolutional Networks](https://aclanthology.org/N18-2078/) *— 2018 · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)*
79. [Fine-grained evaluation of Quality Estimation for Machine translation based on a linguistically motivated Test Suite](https://aclanthology.org/W18-2107/) *— 2018 · Proceedings of the AMTA 2018 Workshop on Translation Quality Estimation and Automatic Post-Editing*
80. [MTNT: A Testbed for Machine Translation of Noisy Text](https://aclanthology.org/D18-1050/) *— 2018 · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing*
81. [Phrase-Based & Neural Unsupervised Machine Translation](https://aclanthology.org/D18-1549/) *— 2018 · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing*
82. [Sentence-Level Fluency Evaluation: References Help, But Can Be Spared!](https://aclanthology.org/K18-1031/) *— 2018 · Proceedings of the 22nd Conference on Computational Natural Language Learning*
83. [Stress Test Evaluation for Natural Language Inference](https://aclanthology.org/C18-1198/) *— 2018 · Proceedings of the 27th International Conference on Computational Linguistics*
84. [TQ-AutoTest – An Automated Test Suite for (Machine) Translation Quality](https://aclanthology.org/L18-1142/) *— 2018 · Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)*
85. [The Other Side of the Coin: Unsupervised Disambiguation of Potentially Idiomatic Expressions by Contrasting Senses](https://aclanthology.org/W18-4919/) *— 2018 · Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018)*
86. [Évaluation morphologique pour la traduction automatique : adaptation au français (Morphological Evaluation for Machine Translation : Adaptation to French)](https://aclanthology.org/2018.jeptalnrecital-long.5/) *— 2018 · Actes de la Conférence TALN. Volume 1 - Articles longs, articles courts de TALN*
87. [Evaluating Discourse Phenomena in Neural Machine Translation](https://arxiv.org/abs/1711.00513) *— 2017 · arXiv:1711.00513*
88. [Evaluating the morphological competence of Machine Translation Systems](https://aclanthology.org/W17-4705/) *— 2017 · Proceedings of the Second Conference on Machine Translation*
89. ballier-2023-translating-dislocations-or-parentheticals-investigating
90. bawden-2018-detecting-context-dependent-sentences-parallel
91. benard-2023-utiliser-les-syntagmes-nominaux-complexes
92. brun-2018-transfert-de-ressources-semantiques-pour
93. cabeca-2023-quality-fit-purpose-building-business
94. dankers-2026-memorisation-meets-compositionality-natural-language
95. kabir-2025-n-gram-attention-how-model
96. kalikman-2026-augmenting-text-increase-translation-difficulty
97. knowles-2023-long-reign-over-us-case
98. laban-2022-near-negative-distinction-giving-second
99. langlais-2017-users-data-two-neglected-children
100. lerner-2024-vers-la-traduction-automatique-des

Pages: **1** · [2](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/isabelle-2017-challenge-set-approach-evaluating-machine-2/)

- [← previous Who cites «Transfer and MT Modularity»](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/isabelle-1986-transfer-mt-modularity/)
- [next → Who cites «A Challenge Set Approach to Evaluating Machine Translation» · page 2](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/isabelle-2017-challenge-set-approach-evaluating-machine-2/)
- [↑ back to papers/cited-by/](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/)