[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/konstas-2017-neural-amr-sequence-sequence-models/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/mtrans/) / Neural AMR: Sequence-to-Sequence Models for Parsing and Generation

# Neural AMR: Sequence-to-Sequence Models for Parsing and Generation

**Authors:** Ioannis Konstas, Srinivasan Iyer, [Mark Yatskar](https://md.mova.elamur.ai/free/en/docs/papers/authors/mark-yatskar/), [Yejin Choi](https://md.mova.elamur.ai/free/en/docs/papers/authors/yejin-choi/), [Luke Zettlemoyer](https://md.mova.elamur.ai/free/en/docs/papers/authors/luke-zettlemoyer/) · **Year:** 2017 · **Venue:** ACL 2017 (Volume 1: Long Papers) **Link:** [aclanthology.org/P17-1014](https://aclanthology.org/P17-1014/) · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)

[**255** citations](https://openalex.org/works?filter=cites:W2610308073) openalex · 2026-10-18[**173** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/konstas-2017-neural-amr-sequence-sequence-models/) of the 140673 papers whose bibliographies we read[Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://aclanthology.org/volumes/P17-1/)[licence](https://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://aclanthology.org/P17-1014/)
- [PDF](https://aclanthology.org/P17-1014.pdf)
- [DOI 10.18653/v1/p17-1014](https://doi.org/10.18653/v1/p17-1014)
- [arXiv 1704.08381](https://arxiv.org/abs/1704.08381)
- [OpenAlex](https://openalex.org/W2610308073)

**Authors (5)**

1. [Ioannis Konstas](https://openalex.org/A5030546839) h 24 · 3660 citations · 111 works
2. [Srinivasan Iyer](https://openalex.org/A5088004913) [ORCID](https://orcid.org/0000-0002-6186-2603) · h 27 · 4155 citations · 91 works
3. [Mark Yatskar](https://openalex.org/A5039205261) h 29 · 7636 citations · 81 works
4. [Yejin Choi](https://openalex.org/A5102992157) [ORCID](https://orcid.org/0000-0003-3032-5378) · h 88 · 36865 citations · 437 works
5. [Luke Zettlemoyer](https://openalex.org/A5067919401) [ORCID](https://orcid.org/0009-0008-8296-0764) · h 93 · 82307 citations · 519 works

Citations and author metrics from openalex (2026-10-18); authors with the same name are not always told apart.

## What it is about

The paper applies sequence-to-sequence neural models to both AMR parsing and AMR generation. The main obstacle is data sparsity, so the authors pretrain on millions of unlabelled Gigaword sentences: a parser labels them, a generator is pretrained on those pairs, and then both are fine-tuned on the 16,833 gold pairs. The graph is linearised by depth-first traversal, with variables and sense tags removed, named entities replaced by indexed anonymous tokens, and scope brackets around subtrees. On the test set the generator reaches 33.8 BLEU and the parser 62.1 Smatch. In the dev ablation, scope markers and anonymisation carry most of the gain, from 21.8 BLEU for the full model to 18.7 with both removed. A per-example random child order performs almost as well as a fixed global one, which suggests the model ignores order artifacts.

## Ideas

- I1. Anonymise named entities as indexed tokens, record the mapping from token to surface name on the training side, and on output put back the most frequent name seen in training. Copy from the input when the name is new. This is a count-based treatment of rare names for Bible-English.
- I2. Scope markers: wrap a child subtree in brackets when its node has more than one child, so the linear form keeps the tree. This is the largest ablation effect in the paper. Test the same markers in the input of `en/src/lin.rs`, measured in residual bits.
- I3. Order control: run the reader with a global random order of edge labels and with a per-example random order, and check that the output does not move. The paper finds the seq2seq model largely agnostic to order; if the translator is not, the order is leaking into the result.
- I4. The paired-training idea has a count-based form: parse a large unlabelled text with the Mova stack, keep sentences whose round trip agrees, and add their meta-worlds as counts. Measure on dev only, and run twice.

## For Mova

The anonymisation idea fits `tools/translate/meta.rs`; the scope markers fit `en/src/lin.rs`; the order control fits the roundtrip tool. The neural models are not for Mova. The principle to keep is that the linearisation should be checked against a random-order control.

Topics: [Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/topics/machine-translation/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.18653/v1/p17-1014](https://doi.org/10.18653/v1/p17-1014) · [arXiv 1704.08381](https://arxiv.org/abs/1704.08381)

```
@inproceedings{konstas2017neural,
  title = {{Neural AMR: Sequence-to-Sequence Models for Parsing and Generation}},
  author = {Konstas, Ioannis and Iyer, Srinivasan and Yatskar, Mark and Choi, Yejin and Zettlemoyer, Luke},
  booktitle = {Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2017},
  doi = {10.18653/v1/p17-1014},
  eprint = {1704.08381},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/p17-1014}
}
```

- [← previous Semi-automatic error analysis for large-scale statistical machine translation](https://md.mova.elamur.ai/free/en/docs/papers/kirchhoff-2007-semi-automatic-error-analysis-large/)
- [next → Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://md.mova.elamur.ai/free/en/docs/papers/krishna-2020-reformulating-unsupervised-style-transfer-as/)
- [↑ back to Machine translation](https://md.mova.elamur.ai/free/en/docs/papers/mtrans/)

## References

34 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. representation. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, Valencia, Spain, pages 536– 546. http://www.aclweb.org/anthology/E17-1051. [↗](https://aclanthology.org/E17-1051/)
2. Yoav Artzi, Kenton Lee, and Luke Zettlemoyer. 2015. Broad-coverage CCG semantic parsing with AMR. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Lisbon, Portugal, pages 1699–1710. http://aclweb.org/anthology/D15-1198. [↗](https://aclanthology.org/D15-1198/)
3. Jenny Rose Finkel, Trond Grenager, and Christopher Manning. 2005. Incorporating non-local information into information extraction systems by Gibbs sampling. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, Ann Arbor, Michigan, pages 363–370. https://doi.org/10.3115/1219840.1219885. [↗](https://doi.org/10.3115/1219840.1219885)
4. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the 2015 International Conference on Learning Representations. CBLS, San Diego, California. http://arxiv.org/abs/1409.0473. [↗](https://arxiv.org/abs/1409.0473)
5. Jeffrey Flanigan, Chris Dyer, Noah A. Smith, and Jaime Carbonell. 2016. Generation from abstract meaning representation using tree transducers. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, San Diego, California, pages 731–739. http://www.aclweb.org/anthology/N16-1087. [↗](https://aclanthology.org/N16-1087/) [our card](https://md.mova.elamur.ai/free/en/docs/papers/flanigan-2016-generation-abstract-meaning-representation-tree/)
6. Guntis Barzdins and Didzis Gosko. 2016. RIGA at SemEval-2016 Task 8: Impact of Smatch extensions and character-level neural translation on AMR parsing accuracy. In Proceedings of the 10th International Workshop on Semantic Evaluation. Association for Computational Linguistics, San Diego, California, pages 1143–1147. http://www.aclweb.org/anthology/S16-1176. [↗](https://aclanthology.org/S16-1176/)
7. Jeffrey Flanigan, Sam Thomson, Jaime Carbonell, Chris Dyer, and Noah A. Smith. 2014. A discriminative graph-based parser for the abstract meaning representation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Baltimore, Maryland, pages 1426–1436. http://www.aclweb.org/anthology/P14-1134. [↗](https://aclanthology.org/P14-1134/)
8. Emily M. Bender. 2014. Language CoLLAGE: Grammatical description with the LinGO grammar matrix. In Proceedings of the 9th International Conference on Language Resources and Evaluation. Reykjavik, Iceland, pages 2447–2451. [find](https://scholar.google.com/scholar?q=Emily+M%2E+Bender%2E+2014%2E+Language+CoLLAGE%3A+Grammatical+description+with+the+LinGO+grammar+matrix%2E+In+Proceedings+of+the+9th+International+Conference+on+Language+Resources+and+Evaluation%2E+Reykjavik%2C+Icel)
9. Johannes Bjerva, Johan Bos, and Hessel Haagsma. 2016. The Meaning Factory at SemEval-2016 Task 8: Producing AMRs with Boxer. In Proceedings of the 10th International Workshop on Semantic Evaluation. Association for Computational Linguistics, San Diego, California, pages 1179–1184. http://www.aclweb.org/anthology/S16-1182. [↗](https://aclanthology.org/S16-1182/)
10. James Goodman, Andreas Vlachos, and Jason Naradowsky. 2016. UCL+Sheffield at SemEval-2016 Task 8: Imitation learning for AMR parsing with an alpha-bound. In Proceedings of the 10th International Workshop on Semantic Evaluation. Association for Computational Linguistics, San Diego, California, pages 1167–1172. http://www.aclweb.org/anthology/S16-1180. [↗](https://aclanthology.org/S16-1180/)
11. Lauritz Brandt, David Grimm, Mengfei Zhou, and Yannick Versley. 2016. ICL-HD at SemEval-2016 Task 8: Meaning representation parsing - augmenting AMR parsing with a preposition semantic role labeling neural network. In Proceedings of the 10th International Workshop on Semantic Evaluation. Association for Computational Linguistics, San Diego, California, pages 1160–1166. http://www.aclweb.org/anthology/S16-1179. [↗](https://aclanthology.org/S16-1179/)
12. Lifu Huang, Taylor Cassidy, Xiaocheng Feng, Heng Ji, Clare R. Voss, Jiawei Han, and Avirup Sil. 2016. Liberal event extraction and event schema induction. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Berlin, Germany, pages 258–268. http://www.aclweb.org/anthology/P16-1025. [↗](https://aclanthology.org/P16-1025/)
13. Shu Cai and Kevin Knight. 2013. Smatch: an evaluation metric for semantic feature structures. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Sofia, Bulgaria, pages 748–752. http://www.aclweb.org/anthology/P13-2131. [↗](https://aclanthology.org/P13-2131/)
14. Bevan Jones, Jacob Andreas, Daniel Bauer, Karl Moritz Hermann, and Kevin Knight. 2012. Semantics-Based Machine Translation with Hyperedge Replacement Grammars. In Proceedings of the 2012 International Conference on Computational Linguistics. Bombay, India, pages 1359–1376. http://www.aclweb.org/anthology/C12-1083. [↗](https://aclanthology.org/C12-1083/) [our card](https://md.mova.elamur.ai/free/en/docs/papers/jones-2012-semantics-based-machine-translation-hyperedge/)
15. Ann Copestake, Dan Flickinger, Carl Pollard, and Ivan A. Sag. 2005. Minimal Recursion Semantics: An introduction. Research on Language and Computation 3(2):281–332. https://doi.org/10.1007/s11168-006-6327-9. [↗](https://doi.org/10.1007/s11168-006-6327-9)
16. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine Marco Damonte, Shay B. Cohen, and Giorgio Satta. 2017. An incremental parser for abstract meaning translation. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Prague, Czech Republic, pages 177–180. http://dl.acm.org/citation.cfm?id=1557769.1557821. abstract meaning representation graphs. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Doha, Qatar, pages 425– 429. http://www.aclweb.org/anthology/D14-1048. [↗](https://aclanthology.org/D14-1048/) [our card](https://md.mova.elamur.ai/free/en/docs/papers/pourdamghani-2014-aligning-english-strings-abstract-meaning/)
17. Fei Liu, Jeffrey Flanigan, Sam Thomson, Norman Sadeh, and Noah A. Smith. 2015. Toward abstractive summarization using semantic representations. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, Denver, Colorado, pages 1077–1086. http://www.aclweb.org/anthology/N15-1114. [↗](https://aclanthology.org/N15-1114/)
18. Nima Pourdamghani, Kevin Knight, and Ulf Hermjakob. 2016. Generating English from abstract meaning representations. In Proceedings of the 9th International Natural Language Generation conference. Association for Computational Linguistics, Edinburgh, UK, pages 21–25. http://anthology.aclweb.org/W16-6603. [↗](https://aclanthology.org/W16-6603/) [our card](https://md.mova.elamur.ai/free/en/docs/papers/pourdamghani-2016-generating-english-abstract-meaning-representations/)
19. Michael Pust, Ulf Hermjakob, Kevin Knight, Daniel Marcu, and Jonathan May. 2015. Parsing english into abstract meaning representation using syntax-based machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Lisbon, Portugal, pages 1143– 1154. https://aclweb.org/anthology/D/D15/D15-1136. [↗](https://aclanthology.org/D15-1136/)
20. Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Lisbon, Portugal, pages 1412– 1421. http://aclweb.org/anthology/D15-1166. [↗](https://aclanthology.org/D15-1166/)
21. Dipendra Kumar Misra and Yoav Artzi. 2016. Neural shift-reduce CCG semantic parsing. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 1775–1786. https://aclweb.org/anthology/D16-1183. [↗](https://aclanthology.org/D16-1183/)
22. Yevgeniy Puzikov, Daisuke Kawahara, and Sadao Kurohashi. 2016. M2L at SemEval-2016 Task 8: AMR parsing with neural networks. In Proceedings of the 10th International Workshop on Semantic Evaluation. Association for Computational Linguistics, San Diego, California, pages 1154–1159. http://www.aclweb.org/anthology/S16-1178. [↗](https://aclanthology.org/S16-1178/)
23. Courtney Napoles, Matthew Gormley, and Benjamin Van Durme. 2012. Annotated Gigaword. In Proceedings of the Joint Workshop on Automatic Knowledge Base Construction and Web-scale Knowledge Extraction. Association for Computational Linguistics, Montréal, Canada, pages 95–100. http://www.aclweb.org/anthology/W12-3018. [↗](https://aclanthology.org/W12-3018/)
24. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Berlin, Germany, pages 86–96. http://www.aclweb.org/anthology/P16-1009. [↗](https://aclanthology.org/P16-1009/)
25. Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The Proposition Bank: An annotated corpus of semantic roles. Computational Linguistics 31(1):71–106. [http://www.cs.rochester.edu/](http://www.cs.rochester.edu/) gildea/palmer-propbank-cl.pdf. [↗](http://www.cs.rochester.edu/) [our card](https://md.mova.elamur.ai/free/en/docs/papers/palmer-2005-proposition-bank-annotated-corpus-semantic/)
26. Melanie Siegel. 2000. HPSG Analysis of Japanese, Springer Berlin Heidelberg, pages 264–279. [find](https://scholar.google.com/scholar?q=Melanie+Siegel%2E+2000%2E+HPSG+Analysis+of+Japanese%2C+Springer+Berlin+Heidelberg%2C+pages+264%E2%80%93279%2E)
27. Linfeng Song, Yue Zhang, Xiaochang Peng, Zhiguo Wang, and Daniel Gildea. 2016. AMR-to-text generation as a traveling salesman problem. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 2084–2089. https://aclweb.org/anthology/D16-1224. [↗](https://aclanthology.org/D16-1224/)
28. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Philadelphia, Pennsylvania, USA, pages 311–318. https://doi.org/10.3115/1073083.1073135. [↗](https://doi.org/10.3115/1073083.1073135) [our card](https://md.mova.elamur.ai/free/en/docs/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
29. Sho Takase, Jun Suzuki, Naoaki Okazaki, Tsutomu Hirao, and Masaaki Nagata. 2016. Neural headline generation on abstract meaning representation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 1054–1059. https://aclweb.org/anthology/D16-1112. [↗](https://aclanthology.org/D16-1112/)
30. Xiaochang Peng, Chuan Wang, Daniel Gildea, and Nianwen Xue. 2017. Addressing the data sparsity issue in neural AMR parsing. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, Valencia, Spain, pages 366–375. http://www.aclweb.org/anthology/E17-1035. [↗](https://aclanthology.org/E17-1035/)
31. Oriol Vinyals, Ł ukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2015. Grammar as a foreign language. In Proceedings of the Nima Pourdamghani, Yang Gao, Ulf Hermjakob, and Kevin Knight. 2014. Aligning English strings with 28th International Conference on Neural Information Processing Systems, MIT Press, pages 2773– 2781. [http://papers.nips.cc/paper/5635-grammar-as-a-foreign-language.pdf](http://papers.nips.cc/paper/5635-grammar-as-a-foreign-language.pdf). [↗](http://papers.nips.cc/paper/5635-grammar-as-a-foreign-language.pdf)
32. Chuan Wang, Sameer Pradhan, Xiaoman Pan, Heng Ji, and Nianwen Xue. 2016. CAMR at SemEval-2016 Task 8: An extended transition-based AMR parser. In Proceedings of the 10th International Workshop on Semantic Evaluation. Association for Computational Linguistics, San Diego, California, pages 1173–1178. http://www.aclweb.org/anthology/S16-1181. [↗](https://aclanthology.org/S16-1181/)
33. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. CoRR abs/1609.08144. http://arxiv.org/abs/1609.08144. [↗](https://arxiv.org/abs/1609.08144)
34. Junsheng Zhou, Feiyu Xu, Hans Uszkoreit, Weiguang QU, Ran Li, and Yanhui Gu. 2016. AMR parsing with an incremental joint model. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 680–689. https://aclweb.org/anthology/D16-1065. [↗](https://aclanthology.org/D16-1065/)