Elamur: **Mova** · [Parse](https://mova.elamur.ai/free/en/parse/) · [Dictionary](https://md.dictionary.elamur.ai/free/en/) · [Translate](https://md.translate.elamur.ai/free/en/) · [Research](https://md.research.elamur.ai/free/en/) · [MCP](https://mcp.elamur.ai/) · [Groupants](https://md.groupants.elamur.ai/free/en/) [Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/pipeline/) [home](https://md.mova.elamur.ai/free/en/) / pipeline # The pipeline How Mova reads a text and answers from it: the text is cut into tokens, parsed into a Universal Dependencies tree, read into frames, gathered into a world, and the answer comes from that world with its reason. Every step is deterministic and can say why. The way back writes the world as text again, byte for byte, and measures what each layer loses (in detail: [the autoencoder](https://mova.elamur.ai/free/en/autoencoder/)). A large model is only a teacher, in training; it never answers. How all of it is trained and measured: [how Mova is trained](https://md.mova.elamur.ai/free/en/training/). [Diagram](#diagram) [Example](#example) [The way back](#way-back) [The autoencoder →](https://mova.elamur.ai/free/en/autoencoder/) [Training loop](#training) [How Mova is trained →](https://md.mova.elamur.ai/free/en/training/) [Text → tokens](#tokens) [Tokens → tree](#tree) [Tree → frames](#frames) [Frames → world](#world) [World → answer](#answer) [Learning](#learn) [And back](#back) ### Reading: text → answer Text text, every byte [Tokens](#tokens) forms + the gap before each; lemma, tag, features [Tree](#tree) UD tree (CoNLL-U): head and relation per token [Frames](#frames) frames: event, verb class, roles, tense, links [World](#world) entities by id + the line of events [Answer](#answer) an answer, with its reason ### The way back: world → text [World](#world) frames, + residual 1.3–2.1 bits a token [Frames](#frames) tree, + residual 6.0–8.6 bits a token [Tree](#tree) tokens, + residual 1.1–5.1 bits a token [Tokens](#tokens) text, + residual 0–0.09 bits a token Text the same bytes as the input ### Learning: the training loop Between the stages, the data that passes; on the way back, the residual each layer needs to give its input back exactly (bits a token, five corpora, measured 2026-10-18; [the autoencoder](https://mova.elamur.ai/free/en/autoencoder/) explains them). The loop is how the models are trained; a large model is in it only as the teacher. ## One story through every stage Real output of Mova's own tools on one short story · run 2026-10-23 · commit `20e228852` · model en.bin of release 0.4 (sha256 31a93401fe71…). Each step names the command that printed it; the commands and the model are in the [quick start](https://github.com/ElamurAI/mova-free#quick-start). ### The text A three-sentence story and a question about it. ``` Mary went to the kitchen. She picked up the apple. Then Mary went to the garden. Where is the apple? ``` ### [Text → tokens](#tokens) Each sentence is cut into tokens; the gap before every token is kept, so the text can come back byte for byte (`roundtrip probe`). ``` tokens Mary | went | to | the | kitchen | . tokens She | picked | up | the | apple | . tokens Then | Mary | went | to | the | garden | . ``` ### [Tokens → tree](#tree) The tokens' lemmas, parts of speech (UPOS and the Penn tag) and features come with them from the tagger; the tree adds each token's head and relation: a Universal Dependencies tree in CoNLL-U (`en annotate`; the second sentence). `SpaceAfter=No` on the full stop is the parser's view of one sentence alone; the space between sentences is kept by the tokens, as the gap before the next sentence's first word. ``` # text = She picked up the apple. 1 She she PRON PRP Case=Nom|Gender=Fem|Number=Sing|Person=3|PronType=Prs 2 nsubj _ _ 2 picked pick VERB VBD Mood=Ind|Number=Sing|Person=3|Tense=Past|VerbForm=Fin 0 root _ _ 3 up up ADP RP _ 2 compound:prt _ _ 4 the the DET DT Definite=Def|PronType=Art 5 det _ _ 5 apple apple NOUN NN Number=Sing 2 obj _ SpaceAfter=No 6 . . PUNCT . _ 2 punct _ SpaceAfter=No ``` ### [Tree → frames](#frames) The tree is read into frames: the event, who did it, to what, where. «She» is already resolved to Mary (`roundtrip probe`). A `frame` line shows the roles only; the frame also keeps the tense, the words around its roles («to the») and the link to the clause before («Then»), and the `frames` line is the sentence said back from the frame alone, exactly. ``` frame Event Mary go place=kitchen frames Mary went to the kitchen. frame Event Mary(She) pick up patient=apple frames She picked up the apple. frame Event Mary go place=garden frames Then Mary went to the garden. ``` ### [Frames → world](#world) The frames of the story become one world: its entities, and the line of its events with roles as ids (`roundtrip probe`). The question reader keeps the state it answers from: where each person is, who holds what, where each thing has been (`world babi-probe --world`). ``` world 4 entities, 3 events entity Mary: named, singular, she entity apple: common, singular entity garden: common, singular entity kitchen: common, singular event 1: Event go agent=Mary place=kitchen event 2: Event pick up agent=Mary patient=apple event 3: Event go agent=Mary place=garden mary is in the garden mary carries apple apple is held by mary apple has been in: kitchen → garden ``` ### [World → answer](#answer) The question is answered from the world, with its reason: the reason names the apple's holder («apple in mary»), and where the holder is comes from the world's state («mary is in the garden», above), so the apple is in the garden (`world babi-probe`). ``` ? Where is the apple? → ("garden", "apple in mary") ``` ### [And back](#back) The world is written back as text. Without its residual, the third sentence comes back with «she» for «Mary»; the residual holds that one item, a 1-bit flag (say the name, which the world has), and with it the world gives every frame back exactly. The tokens, the tree and the frames of these three sentences need no residual (each of their lines above says its sentence back as it was); in longer texts they do, as the table below shows (`roundtrip probe`). ``` residual world pron/name: 1 items, 1 bits: agent Mary: the name, where the generator would say «She» (1 bit: a flag) exact world + residual → frames: equal world → «Mary went to the kitchen.» (text «Mary went to the kitchen.») world → «She picked up the apple.» (text «She picked up the apple.») world → «Then she went to the garden.» (text «Then Mary went to the garden.») ``` ## The way back: world → frames → tree → text Mova does not only read. Every layer has a decoder that goes the other way: the world is said as frames (a referring-expression generator chooses a name or a pronoun), the frames are realized as a tree (the verb group from the frame's tense, aspect, voice and modality; the roles in their places with their prepositions), the tree is ordered and inflected into tokens, and the tokens are joined into text. Whatever a layer cannot give back is stored beside it as a **residual**, put in a class (order, punctuation, a pronoun or a name, a dropped word, …) and counted in bits. With every residual, whole books come back **byte for byte**; that is the gate. In detail, with every class, gate and control and what the residual asks for next: [the autoencoder](https://mova.elamur.ai/free/en/autoencoder/). **Why it matters.** The residual is an honest measure of each layer: what it does not describe shows up as bits, by class, with examples. It ranks the next change to make, and a change that does not shrink the residual has not helped. It also keeps the reader honest: a world that only looks right cannot give its text back. | layer | residual, bits a token | what it is mostly | | --- | --- | --- | | text ↔ tokens | 0–0.09 | line wraps of a book | | tokens ↔ tree | 1.1–5.1 | word order, typography, contractions | | tree ↔ frames | 6.0–8.6 | punctuation, words the frames do not keep, order | | frames ↔ world | 1.3–2.1 | which pronoun, links to frames the world drops | Measured 2026-10-18, a few days before the example above, on five sets (23,586 school lessons, the agreed school treebank (school sentences the parsers and the teacher agreed on, partly the parser's own training data), 41 tale books, the Torah, *Alice's Adventures in Wonderland*); every volume came back byte for byte, and two runs gave identical reports. Method, classes and the full tables: [`tools/roundtrip`](https://md.mova.elamur.ai/free/en/docs/tools/roundtrip/). In the example above, the world needed one item of residual, a 1-bit flag: the name where the generator would say «she». This table has no baseline yet (what a general compressor needs for the same text), so it ranks the layers rather than grading them. **The weakest link.** Tree ↔ frames leaves the most behind, 6.0–8.6 bits a token, against 1.1–5.1 for the tree below it and 1.3–2.1 for the world above: the frames keep the events and their roles but not yet the punctuation, the function words around them or their order. By the rule above it is the next target. ## Training: a loop The small models learn round after round (the loop in the [diagram](#diagram); code: [`tools/cycle`](https://md.mova.elamur.ai/free/en/docs/tools/cycle/)): 1. **Drafts.** Mova's own small model parses new sentences: the draft tree. 2. **Witnesses.** Open neural parsers parse the same sentences. 3. **Agreement.** [`treeagree`](https://md.mova.elamur.ai/free/en/docs/tools/treeagree/) votes the draft with the witnesses into one agreed tree, with its features and lemmas. 4. **The teacher checks.** A large model, as teacher, gets the agreed draft (never a bare sentence) and corrects it; every correction passes the UD gates, and the open parsers check it again. 5. **The parse database.** The checked trees join the corpus of agreed trees. 6. **The trainer.** The small model is retrained on it (a warm start, every round) and drafts the next round. **Where a large model touches the pipeline.** At two points, both in training: as the **teacher of trees** (step 4 above), and as the **teacher of pragmatics**, writing silver labels for the pragmatics classifiers, which pass their own gates before anything learns from them. It never answers: at answer time every stage above is Mova's own deterministic code and its trained small models, and the reason given with an answer comes from the world, not from a model's text. ## Text → tokens The text is cut into sentences and tokens, every space and mark kept, and each token gets its lemma, part of speech and features. **takes**text: every byte of it **gives**tokens: the forms, the gap before each, the sentence breaks; each with its lemma, part of speech and features **code**[`en/src/tok.rs`](https://md.mova.elamur.ai/free/en/docs/en/) [`en/src/tag.rs`](https://md.mova.elamur.ai/free/en/docs/en/) [`en/src/ptag.rs`](https://md.mova.elamur.ai/free/en/docs/en/) Measure: the text back byte for byte from the tokens: 0–0.09 bits a token of residual (2026-10-18); token, sentence-boundary and tagging accuracy not yet reported ### Implements 2 · the code stands on it, or tried it - [TnT – A Statistical Part-of-Speech Tagger](https://research.elamur.ai/free/en/papers/brants-2000-tnt-statistical-part-speech-tagger/) — Brants 2000 [measured, gain](https://md.mova.elamur.ai/free/en/docs/papers/measured/#brants-2000-tnt-statistical-part-speech-tagger) - [A Good Part-of-Speech Tagger in about 200 Lines of Python](https://research.elamur.ai/free/en/papers/honnibal-2013-good-part-of-speech-tagger/) — Honnibal 2013 ### Source of rules / background 2 · hand-written rules and knowledge taken from it, or reading behind a decision - [Part-of-Speech Tagging Guidelines for the Penn Treebank Project (3rd Revision)](https://research.elamur.ai/free/en/papers/santorini-1990-part-of-speech-tagging-guidelines-penn-treebank/) — Santorini 1990 rule source - [Unsupervised Multilingual Sentence Boundary Detection (Punkt)](https://research.elamur.ai/free/en/papers/kiss-2006-unsupervised-multilingual-sentence-boundary-detection/) — Kiss 2006 ## Tokens → tree Each sentence becomes a dependency tree in the style of Universal Dependencies: who depends on whom, and how. A grammar expert system checks the tree with hand-written rules. **takes**tokens **gives**a Universal Dependencies tree per sentence (CoNLL-U): head and relation of every token **code**[`en/src/parse.rs`](https://md.mova.elamur.ai/free/en/docs/en/) [`en/src/expert.rs`](https://md.mova.elamur.ai/free/en/docs/en/) [`gparse`](https://md.mova.elamur.ai/free/en/docs/gparse/) Measure: LAS on held-out treebanks: 85.7 on EWT (the English Web Treebank), 88.5 on the school's agreed test (2026-10-15) **Old grammars as a source of rules.** Many of the rules of Mova's grammar expert system come from grammars written between 1851 and 1931 — Brown 1851, Mätzner 1874, Reed 1877, Whitney 1877, Sweet 1891, Poutsma 1904, Jespersen 1914, Poutsma 1921, Jespersen 1924, Kruisinga 1925, Fowler 1926, Curme 1931 — rewritten as checks over Universal Dependencies trees, each with its source, and verified on gold treebanks. It is a rare choice: these books describe English in more detail than any treebank guideline, and what they say can be read, cited and tested. ### Implements 3 · the code stands on it, or tried it - [Incremental Parsing with the Perceptron Algorithm](https://research.elamur.ai/free/en/papers/collins-2004-incremental-parsing-perceptron-algorithm/) — Collins 2004 [measured, compared](https://md.mova.elamur.ai/free/en/docs/papers/measured/#collins-2004-incremental-parsing-perceptron-algorithm) - [Incrementality in Deterministic Dependency Parsing](https://research.elamur.ai/free/en/papers/nivre-2004-incrementality-deterministic-dependency-parsing/) — Nivre 2004 - [A Tale of Two Parsers: Investigating and Combining Graph-based and Transition-based Dependency Parsing](https://research.elamur.ai/free/en/papers/zhang-2008-tale-two-parsers-investigating-combining/) — Zhang 2008 [measured, gain](https://md.mova.elamur.ai/free/en/docs/papers/measured/#zhang-2008-tale-two-parsers-investigating-combining) ### Measured against 1 · benchmarks, metrics, scoring conventions, reference results - [Applying Occam's Razor to Transformer-Based Dependency Parsing: What Works, What Doesn't, and What is Really Necessary](https://research.elamur.ai/free/en/papers/grunewald-2021-applying-occam-s-razor-transformer/) — Grünewald 2021 [measured, compared](https://md.mova.elamur.ai/free/en/docs/papers/measured/#grunewald-2021-applying-occam-s-razor-transformer) ### Source of rules / background 48 · hand-written rules and knowledge taken from it, or reading behind a decision - [Universal Dependencies for Learner English](https://research.elamur.ai/free/en/papers/berzak-2016-universal-dependencies-learner-english/) — Berzak 2016 rule source - [The Grammar of English Grammars](https://research.elamur.ai/free/en/papers/brown-1851-grammar-english-grammars/) — Brown 1851 rule source - [Automatic Annotation and Evaluation of Error Types for Grammatical Error Correction](https://research.elamur.ai/free/en/papers/bryant-2017-automatic-annotation-evaluation-error-types/) — Bryant 2017 rule source - [«Are you Afraid of Ghosts?» A Proposal for Busting Predicate Ellipsis in Universal Dependencies](https://research.elamur.ai/free/en/papers/corbetta-2025-guillemotleftare-you-afraid-ghosts-guillemotright/) — Corbetta 2025 rule source - [A Grammar of the English Language, Vol. III: Syntax](https://research.elamur.ai/free/en/papers/curme-1931-grammar-english-language-vol/) — Curme 1931 rule source - [Identifying and Handling Cross-Treebank Inconsistencies in UD: A Pilot Study](https://research.elamur.ai/free/en/papers/donicke-2020-identifying-handling-cross-treebank-inconsistencies/) — Dönicke 2020 rule source - [Negation in Universal Dependencies](https://research.elamur.ai/free/en/papers/findlay-2025-negation-universal-dependencies/) — Findlay 2025 rule source - [A Dictionary of Modern English Usage](https://research.elamur.ai/free/en/papers/fowler-1926-dictionary-modern-english-usage/) — Fowler 1926 rule source - [The Cambridge Grammar of the English Language](https://research.elamur.ai/free/en/papers/huddleston-2002-cambridge-grammar-english-language/) — Huddleston 2002 rule source - [A Modern English Grammar on Historical Principles, Parts II–VI](https://research.elamur.ai/free/en/papers/jespersen-1914-modern-english-grammar-historical/) — Jespersen 1914 rule source - [The Philosophy of Grammar](https://research.elamur.ai/free/en/papers/jespersen-1924-philosophy-grammar/) — Jespersen 1924 rule source - [A Handbook of Present-Day English](https://research.elamur.ai/free/en/papers/kruisinga-1925-handbook-present-day-english/) — Kruisinga 1925 rule source - [Annotating Second Language in Universal Dependencies: a Review of Current Practices and Directions for Harmonized Guidelines](https://research.elamur.ai/free/en/papers/masciolini-2025-annotating-second-language-universal-dependencies/) — Masciolini 2025 rule source - [An English Grammar: Methodical, Analytical, and Historical](https://research.elamur.ai/free/en/papers/matzner-1874-english-grammar-methodical-analytical/) — Mätzner 1874 rule source - [A Grammar of Late Modern English](https://research.elamur.ai/free/en/papers/poutsma-1904-grammar-late-modern-english/) — Poutsma 1904 rule source - [The Characters of the English Verb (1921); The Infinitive, the Gerund and the Participles of the English Verb (1923)](https://research.elamur.ai/free/en/papers/poutsma-1921-characters-english-verb/) — Poutsma 1921 rule source - [Higher Lessons in English (1877/1886) and Graded Lessons in English (1877)](https://research.elamur.ai/free/en/papers/reed-1877-higher-lessons-english/) — Reed 1877 rule source - [Mischievous nominal constructions in Universal Dependencies](https://research.elamur.ai/free/en/papers/schneider-2021-mischievous-nominal-constructions-universal-dependencies/) — Schneider 2021 rule source - [A New English Grammar, Logical and Historical](https://research.elamur.ai/free/en/papers/sweet-1891-new-english-grammar-logical/) — Sweet 1891 rule source - [Essentials of English Grammar for the Use of Schools](https://research.elamur.ai/free/en/papers/whitney-1877-essentials-english-grammar-use/) — Whitney 1877 rule source **Background reading: 28 works** - [Estimating POS Annotation Consistency of Different Treebanks in a Language](https://research.elamur.ai/free/en/papers/aggarwal-2020-estimating-pos-annotation-consistency-different/) — Aggarwal 2020 - [The BEA-2019 Shared Task on Grammatical Error Correction](https://research.elamur.ai/free/en/papers/bryant-2019-bea-2019-shared-task-grammatical/) — Bryant 2019 - [Classifying Syntactic Errors in Learner Language](https://research.elamur.ai/free/en/papers/choshen-2020-classifying-syntactic-errors-learner-language/) — Choshen 2020 - [Syntax of referents of relative markers: Evidence from a corpus of learner English](https://research.elamur.ai/free/en/papers/czerniak-2025-syntax-referents-relative-markers-evidence/) — Czerniak 2025 - [ICLE-RC: International Corpus of Learner English for Relative Clauses](https://research.elamur.ai/free/en/papers/das-2025-icle-rc-international-corpus-learner/) — Das 2025 - [Universal Dependencies](https://research.elamur.ai/free/en/papers/demarneffe-2021-universal-dependencies/) — de Marneffe 2021 - [Correcting Dependency Annotation Errors](https://research.elamur.ai/free/en/papers/dickinson-2009-correcting-dependency-annotation-errors/) — Dickinson 2009 - [On Grammaticality in the Syntactic Annotation of Learner Language](https://research.elamur.ai/free/en/papers/dickinson-2015-grammaticality-syntactic-annotation-learner-language/) — Dickinson 2015 - [Assessing the Agreement Competence of Large Language Models](https://research.elamur.ai/free/en/papers/garcia-2025-assessing-agreement-competence-large-language/) — García 2025 - [LLM Dependency Parsing with In-Context Rules](https://research.elamur.ai/free/en/papers/ginn-2025-llm-dependency-parsing-context-rules/) — Ginn 2025 - [Coordinate Constructions in English Enhanced Universal Dependencies: Analysis and Computational Modeling](https://research.elamur.ai/free/en/papers/grunewald-2021-coordinate-constructions-english-enhanced-universal/) — Grünewald 2021 - [Collocational bootstrapping: A hypothesis about the learning of subject-verb agreement in humans and neural networks](https://research.elamur.ai/free/en/papers/hobbs-2026-collocational-bootstrapping-hypothesis-about-learning/) — Hobbs 2026 - [Minor changes make a difference: a case study on the consistency of UD-based dependency parsers](https://research.elamur.ai/free/en/papers/kalpakchi-2021-minor-changes-make-difference-case/) — Kalpakchi 2021 - [Targeted Syntactic Evaluation for Grammatical Error Correction](https://research.elamur.ai/free/en/papers/koyama-2025-targeted-syntactic-evaluation-grammatical-error/) — Koyama 2025 - [A Dependency Treebank of Spoken Second Language English](https://research.elamur.ai/free/en/papers/kyle-2022-dependency-treebank-spoken-second-language/) — Kyle 2022 - [Annotating Coordination in the Penn Treebank](https://research.elamur.ai/free/en/papers/maier-2012-annotating-coordination-penn-treebank/) — Maier 2012 - [The Stanford Typed Dependencies Representation](https://research.elamur.ai/free/en/papers/marneffe-2008-stanford-typed-dependencies-representation/) — de Marneffe 2008 - [Assessing the Annotation Consistency of the Universal Dependencies Corpora](https://research.elamur.ai/free/en/papers/marneffe-2017-assessing-annotation-consistency-universal-dependencies/) — de Marneffe 2017 - [Step-by-step Instructions and a Simple Tabular Output Format Improve the Dependency Parsing Accuracy of LLMs](https://research.elamur.ai/free/en/papers/matsuda-2025-step-step-instructions-simple-tabular/) — Matsuda 2025 - [Probing the Dynamics of Syntactic Ability Acquisition Throughout LLM Pretraining](https://research.elamur.ai/free/en/papers/matsuda-2026-probing-dynamics-syntactic-ability-acquisition/) — Matsuda 2026 - [All Roads Lead to UD: Converting Stanford and Penn Parses to English Universal Dependencies with Multilayer Annotations](https://research.elamur.ai/free/en/papers/peng-2018-all-roads-lead-ud-converting/) — Peng 2018 - [Conjunct Lengths in English, Dependency Length Minimization, and Dependency Structure of Coordination](https://research.elamur.ai/free/en/papers/przepiorkowski-2023-conjunct-lengths-english-dependency-length/) — Przepiórkowski 2023 - [Does Universal Dependencies need a parsing representation? An investigation of English](https://research.elamur.ai/free/en/papers/silveira-2015-does-universal-dependencies-need-parsing/) — Silveira 2015 - [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://research.elamur.ai/free/en/papers/warstadt-2020-blimp-benchmark-linguistic-minimal-pairs/) — Warstadt 2020 - [Subject-verb agreement with Seq2Seq transformers: Bigger is better, but still not best](https://research.elamur.ai/free/en/papers/wilson-2023-subject-verb-agreement-seq2seq-transformers/) — Wilson 2023 - [Assessing the influence of attractor-verb distance on grammatical agreement in humans and language models](https://research.elamur.ai/free/en/papers/zacharopoulos-2023-assessing-influence-attractor-verb-distance/) — Zacharopoulos 2023 - [Are UD Treebanks Getting More Consistent? A Report Card for English UD](https://research.elamur.ai/free/en/papers/zeldes-2023-are-ud-treebanks-getting-more/) — Zeldes 2023 - [Encoding Logical Relations of Chinese Complex Sentences within the Universal Dependencies Framework](https://research.elamur.ai/free/en/papers/zhu-2026-encoding-logical-relations-chinese-complex/) — Zhu 2026 ## Tree → frames The tree is read into frames: who did what to whom, when, where, why, with the speech act and the VerbNet class; pronouns are resolved to what they stand for. **takes**trees, and the discourse so far **gives**frames: the event, its verb and VerbNet class, its roles (agent, patient, recipient, place …), tense and modality, the links between clauses; pronouns resolved **code**[`prag/src/frames.rs`](https://md.mova.elamur.ai/free/en/docs/prag/) [`prag/src/vn.rs`](https://md.mova.elamur.ai/free/en/docs/prag/) [`coref`](https://md.mova.elamur.ai/free/en/docs/coref/) Measure: frame F1 against agreed frames: not yet measured (no agreed frames yet) ### Implements 8 · the code stands on it, or tried it - [CogNIAC: A Discourse Processing Engine](https://research.elamur.ai/free/en/papers/baldwin-1995-cogniac-discourse-processing-engine/) — Baldwin 1995 - [CogNIAC: high precision coreference with limited knowledge and linguistic resources](https://research.elamur.ai/free/en/papers/baldwin-1997-cogniac-high-precision-coreference-limited/) — Baldwin 1997 - [Simple Coreference Resolution with Rich Syntactic and Semantic Features](https://research.elamur.ai/free/en/papers/haghighi-2009-simple-coreference-resolution-rich-syntactic/) — Haghighi 2009 - [Resolving Pronoun References](https://research.elamur.ai/free/en/papers/hobbs-1978-resolving-pronoun-references/) — Hobbs 1978 [measured, gain](https://md.mova.elamur.ai/free/en/docs/papers/measured/#hobbs-1978-resolving-pronoun-references) - [Deterministic Coreference Resolution Based on Entity-Centric, Precision-Ranked Rules](https://research.elamur.ai/free/en/papers/lee-2013-deterministic-coreference-resolution-based-entity/) — Lee 2013 [measured, gain](https://md.mova.elamur.ai/free/en/docs/papers/measured/#lee-2013-deterministic-coreference-resolution-based-entity) - [Udapi: Universal API for Universal Dependencies](https://research.elamur.ai/free/en/papers/popel-2017-udapi-universal-api-universal-dependencies/) — Popel 2017 - [A Multi-Pass Sieve for Coreference Resolution](https://research.elamur.ai/free/en/papers/raghunathan-2010-multi-pass-sieve-coreference-resolution/) — Raghunathan 2010 [measured, gain](https://md.mova.elamur.ai/free/en/docs/papers/measured/#raghunathan-2010-multi-pass-sieve-coreference-resolution) - [VerbNet: A Broad-Coverage, Comprehensive Verb Lexicon](https://verbs.colorado.edu/verbnet/) — Kipper Schuler 2005 ### Measured against 8 · benchmarks, metrics, scoring conventions, reference results - [Algorithms for Scoring Coreference Chains](https://research.elamur.ai/free/en/papers/bagga-1998-algorithms-scoring-coreference-chains/) — Bagga 1998 [measured, the metric](https://md.mova.elamur.ai/free/en/docs/papers/measured/#bagga-1998-algorithms-scoring-coreference-chains) - [On Coreference Resolution Performance Metrics](https://research.elamur.ai/free/en/papers/luo-2005-coreference-resolution-performance-metrics/) — Luo 2005 [measured, the metric](https://md.mova.elamur.ai/free/en/docs/papers/measured/#luo-2005-coreference-resolution-performance-metrics) - [CorefUD 1.0: Coreference Meets Universal Dependencies](https://research.elamur.ai/free/en/papers/nedoluzhko-2022-corefud-1-0-coreference-meets/) — Nedoluzhko 2022 - [Findings of the Fourth Shared Task on Multilingual Coreference Resolution: Can LLMs Dethrone Traditional Approaches?](https://research.elamur.ai/free/en/papers/novak-2025-findings-fourth-shared-task-multilingual/) — Novák 2025 - [Findings of the Fifth Shared Task on Multilingual Coreference Resolution: Expanding Datasets for Long-Range Entities](https://research.elamur.ai/free/en/papers/novak-2026-findings-fifth-shared-task-multilingual/) — Novák 2026 - [Scoring Coreference Partitions of Predicted Mentions: A Reference Implementation](https://research.elamur.ai/free/en/papers/pradhan-2014-scoring-coreference-partitions-predicted-mentions/) — Pradhan 2014 [measured, the metric](https://md.mova.elamur.ai/free/en/docs/papers/measured/#pradhan-2014-scoring-coreference-partitions-predicted-mentions) - [A Model-Theoretic Coreference Scoring Scheme](https://research.elamur.ai/free/en/papers/vilain-1995-model-theoretic-coreference-scoring-scheme/) — Vilain 1995 [measured, the metric](https://md.mova.elamur.ai/free/en/docs/papers/measured/#vilain-1995-model-theoretic-coreference-scoring-scheme) - [The Proposition Bank: An Annotated Corpus of Semantic Roles](https://research.elamur.ai/free/en/papers/palmer-2005-proposition-bank-annotated-corpus-semantic/) — Palmer 2005 ### Source of rules / background 7 · hand-written rules and knowledge taken from it, or reading behind a decision - [Politeness: Some Universals in Language Usage](https://research.elamur.ai/free/en/papers/brown-1987-politeness-some-universals-language-usage/) — Brown 1987 rule source - [An Argument for Basic Emotions](https://research.elamur.ai/free/en/papers/ekman-1992-argument-basic-emotions/) — Ekman 1992 rule source - [Hedging in Scientific Research Articles](https://research.elamur.ai/free/en/papers/hyland-1998-hedging-scientific-research-articles/) — Hyland 1998 rule source [measured, compared](https://md.mova.elamur.ai/free/en/docs/papers/measured/#hyland-1998-hedging-scientific-research-articles) - [Hedges: A Study in Meaning Criteria and the Logic of Fuzzy Concepts](https://research.elamur.ai/free/en/papers/lakoff-1973-hedges-study-meaning-criteria/) — Lakoff 1973 rule source - [Speech Acts: An Essay in the Philosophy of Language](https://research.elamur.ai/free/en/papers/searle-1969-speech-acts/) — Searle 1969 rule source - [An Algorithm for Pronominal Anaphora Resolution](https://research.elamur.ai/free/en/papers/lappin-1994-algorithm-pronominal-anaphora-resolution/) — Lappin 1994 - [The Berkeley FrameNet Project](https://research.elamur.ai/free/en/papers/baker-1998-berkeley-framenet-project/) — Baker 1998 ## Frames → world The frames of a text become one world: its entities, their states and the events between them. **takes**frames **gives**the world: entities by id, and the line of events with roles as ids **code**[`world`](https://md.mova.elamur.ai/free/en/docs/world/) [`prag/src/roles.rs`](https://md.mova.elamur.ai/free/en/docs/prag/) Measure: the world's residual: 1.3–2.1 bits a token over five corpora (2026-10-18) ### Implements 7 · the code stands on it, or tried it - [Categorization and Representation of Physics Problems by Experts and Novices](https://research.elamur.ai/free/en/papers/chi-1981-categorization-representation-physics-problems-experts/) — Chi 1981 - [Learning to Solve Arithmetic Word Problems with Verb Categorization](https://research.elamur.ai/free/en/papers/hosseini-2014-learning-solve-arithmetic-word-problems/) — Hosseini 2014 - [COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs](https://research.elamur.ai/free/en/papers/hwang-2020-comet-atomic-2020-symbolic-neural/) — Hwang 2020 [measured, gain](https://md.mova.elamur.ai/free/en/docs/papers/measured/#hwang-2020-comet-atomic-2020-symbolic-neural) - [Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction](https://research.elamur.ai/free/en/papers/jie-2022-learning-reason-deductively-math-word/) — Jie 2022 [tried, no gain](https://md.mova.elamur.ai/free/en/docs/papers/measured/#jie-2022-learning-reason-deductively-math-word) - [Solving General Arithmetic Word Problems](https://research.elamur.ai/free/en/papers/roy-2015-solving-general-arithmetic-word-problems/) — Roy 2015 [tried, no gain](https://md.mova.elamur.ai/free/en/docs/papers/measured/#roy-2015-solving-general-arithmetic-word-problems) - [Unit Dependency Graph and its Application to Arithmetic Word Problem Solving](https://research.elamur.ai/free/en/papers/roy-2017-unit-dependency-graph-arithmetic/) — Roy 2017 [measured, gain and no gain](https://md.mova.elamur.ai/free/en/docs/papers/measured/#roy-2017-unit-dependency-graph-arithmetic) - [Mapping to Declarative Knowledge for Word Problem Solving](https://research.elamur.ai/free/en/papers/roy-2018-mapping-declarative-knowledge-word-problem/) — Roy 2018 ### Source of rules / background 1 · hand-written rules and knowledge taken from it, or reading behind a decision - [Learning To Use Formulas To Solve Simple Arithmetic Problems](https://research.elamur.ai/free/en/papers/mitra-2016-learning-use-formulas-solve-simple/) — Mitra 2016 rule source ## World → answer A question is read the same way and answered from the world, with the reason for the answer. **takes**the world, and a question read as a frame **gives**an answer, with the reason it holds **code**[`world/src/babi.rs`](https://md.mova.elamur.ai/free/en/docs/world/) [`world/src/qa.rs`](https://md.mova.elamur.ai/free/en/docs/world/) [`qagen`](https://md.mova.elamur.ai/free/en/docs/qagen/) [`math`](https://md.mova.elamur.ai/free/en/docs/math/) Measure: the school's tests: 38.8% right of 49 760, four options each, the strongest control (questions shuffled) 29.6% (2026-10-22) ### Measured against 6 · benchmarks, metrics, scoring conventions, reference results - [Evaluating Models' Local Decision Boundaries via Contrast Sets](https://research.elamur.ai/free/en/papers/gardner-2020-evaluating-models-local-decision-boundaries/) — Gardner 2020 [measured, gain](https://md.mova.elamur.ai/free/en/docs/papers/measured/#gardner-2020-evaluating-models-local-decision-boundaries) - [SPARTQA: A Textual Question Answering Benchmark for Spatial Reasoning](https://research.elamur.ai/free/en/papers/mirzaee-2021-spartqa-textual-question-answering-benchmark/) — Mirzaee 2021 [measured, our score](https://md.mova.elamur.ai/free/en/docs/papers/measured/#mirzaee-2021-spartqa-textual-question-answering-benchmark) - [Are NLP Models really able to Solve Simple Math Word Problems?](https://research.elamur.ai/free/en/papers/patel-2021-are-nlp-models-really-able/) — Patel 2021 - [StepGame: A New Benchmark for Robust Multi-Hop Spatial Reasoning in Texts](https://research.elamur.ai/free/en/papers/shi-2022-stepgame-new-benchmark-robust-multi/) — Shi 2022 [measured, our score](https://md.mova.elamur.ai/free/en/docs/papers/measured/#shi-2022-stepgame-new-benchmark-robust-multi) - [Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks](https://research.elamur.ai/free/en/papers/weston-2015-towards-ai-complete-question-answering/) — Weston 2015 [measured, our score](https://md.mova.elamur.ai/free/en/docs/papers/measured/#weston-2015-towards-ai-complete-question-answering) - [Fantastic Questions and Where to Find Them: FairytaleQA – An Authentic Dataset for Narrative Comprehension](https://research.elamur.ai/free/en/papers/xu-2022-fantastic-questions-where-find-them/) — Xu 2022 ### Source of rules / background 1 · hand-written rules and knowledge taken from it, or reading behind a decision - [A Corpus for Reasoning About Natural Language Grounded in Photographs (NLVR2)](https://research.elamur.ai/free/en/papers/suhr-2019-corpus-reasoning-natural-language-grounded/) — Suhr 2019 ## Learning The small models learn in a loop: they draft, open parsers witness, the drafts are agreed, a teacher checks them, and the models retrain on the result. A gain counts only when two runs give identical numbers and it shows on the two-fold check (each half of the training set held out in turn); the test set runs once per release. **takes**drafts of the small models, open parsers, the teacher's corrections **gives**agreed trees and labels; retrained models **code**[`tools/cycle`](https://md.mova.elamur.ai/free/en/docs/tools/cycle/) [`tools/treeagree`](https://md.mova.elamur.ai/free/en/docs/tools/treeagree/) [`annot`](https://md.mova.elamur.ai/free/en/docs/annot/) [`gparse`](https://md.mova.elamur.ai/free/en/docs/gparse/) [`train`](https://md.mova.elamur.ai/free/en/docs/train/) Measure: how much of the small model's draft the teacher keeps: 98.9 UAS, 98.5 LAS on the newest round of tales (2026-10-23) ### Implements 11 · the code stands on it, or tried it - [Instance-Based Learning Algorithms](https://research.elamur.ai/free/en/papers/aha-1991-instance-based-learning-algorithms/) — Aha 1991 [measured, compared](https://md.mova.elamur.ai/free/en/docs/papers/measured/#aha-1991-instance-based-learning-algorithms) - [Cutting Through the Noise: Boosting LLM Performance on Math Word Problems](https://research.elamur.ai/free/en/papers/anantheswaran-2024-cutting-through-noise-boosting-llm/) — Anantheswaran 2024 [tried, no gain: training on problems with distractors](https://md.mova.elamur.ai/free/en/docs/papers/measured/#anantheswaran-2024-cutting-through-noise-boosting-llm) - [Curriculum Learning](https://research.elamur.ai/free/en/papers/bengio-2009-curriculum-learning/) — Bengio 2009 [tried, no gain](https://md.mova.elamur.ai/free/en/docs/papers/measured/#bengio-2009-curriculum-learning) - [Bagging Predictors](https://research.elamur.ai/free/en/papers/breiman-1996-bagging-predictors/) — Breiman 1996 - [Discriminative Reranking for Natural Language Parsing](https://research.elamur.ai/free/en/papers/collins-2000-discriminative-reranking-natural-language-parsing/) — Collins 2000 [measured, gain](https://md.mova.elamur.ai/free/en/docs/papers/measured/#collins-2000-discriminative-reranking-natural-language-parsing) - [Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms](https://research.elamur.ai/free/en/papers/collins-2002-discriminative-training-methods-hidden-markov/) — Collins 2002 [measured, gain](https://md.mova.elamur.ai/free/en/docs/papers/measured/#collins-2002-discriminative-training-methods-hidden-markov) - [IGTree: Using Trees for Compression and Classification in Lazy Learning Algorithms](https://research.elamur.ai/free/en/papers/daelemans-1997-igtree-trees-compression-classification-lazy/) — Daelemans 1997 [measured, compared](https://md.mova.elamur.ai/free/en/docs/papers/measured/#daelemans-1997-igtree-trees-compression-classification-lazy) - [Structured Perceptron with Inexact Search](https://research.elamur.ai/free/en/papers/huang-2012-structured-perceptron-inexact-search/) — Huang 2012 [tried, no gain](https://md.mova.elamur.ai/free/en/docs/papers/measured/#huang-2012-structured-perceptron-inexact-search) - [C4.5: Programs for Machine Learning](https://research.elamur.ai/free/en/papers/quinlan-1993-c45-programs-machine-learning/) — Quinlan 1993 - [How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark](https://research.elamur.ai/free/en/papers/yang-2025-llm-reasoning-distracted-irrelevant-context/) — Yang 2025 [tried, no gain: training on problems with distractors](https://md.mova.elamur.ai/free/en/docs/papers/measured/#yang-2025-llm-reasoning-distracted-irrelevant-context) - [STaR: Bootstrapping Reasoning With Reasoning](https://research.elamur.ai/free/en/papers/zelikman-2022-star-bootstrapping-reasoning-reasoning/) — Zelikman 2022 [tried, no gain](https://md.mova.elamur.ai/free/en/docs/papers/measured/#zelikman-2022-star-bootstrapping-reasoning-reasoning) ### Measured against 1 · benchmarks, metrics, scoring conventions, reference results - [Assessing the Reliability of Statistical Software: Part I](https://research.elamur.ai/free/en/papers/mccullough-1998-assessing-reliability-statistical-software/) — McCullough 1998 [measured, the metric](https://md.mova.elamur.ai/free/en/docs/papers/measured/#mccullough-1998-assessing-reliability-statistical-software) ### Source of rules / background 3 · hand-written rules and knowledge taken from it, or reading behind a decision - [Parsing the Switch: LLM-Based UD Annotation for Complex Code-Switched and Low-Resource Languages](https://research.elamur.ai/free/en/papers/kellert-2025-parsing-switch-llm-based-ud/) — Kellert 2025 rule source - [Pre-annotation Matters: A Comparative Study on POS and Dependency Annotation for an Alsatian Dialect](https://research.elamur.ai/free/en/papers/bernhard-2025-pre-annotation-matters-comparative-study/) — Bernhard 2025 - [Evaluating large language models for the tasks of PoS tagging within the Universal Dependency framework](https://research.elamur.ai/free/en/papers/machado-2024-evaluating-large-language-models-tasks/) — Machado 2024 ## And back The world is written back into text, byte for byte with its residual: what each layer loses is measured. **takes**the world and its residual **gives**frames, the tree, the tokens and the text again, byte for byte **code**[`tools/roundtrip`](https://md.mova.elamur.ai/free/en/docs/tools/roundtrip/) [`en/src/lin.rs`](https://md.mova.elamur.ai/free/en/docs/en/) [`en/src/morph.rs`](https://md.mova.elamur.ai/free/en/docs/en/) [`tools/translate`](https://md.mova.elamur.ai/free/en/docs/tools/translate/) Measure: residual bits a token at each layer, in the table below (2026-10-18) ****Background reading** 9 · hand-written rules and knowledge taken from it, or reading behind a decision** - [The First Surface Realisation Shared Task: Overview and Evaluation Results](https://research.elamur.ai/free/en/papers/belz-2011-first-surface-realisation-shared-task/) — Belz 2011 - [The First Multilingual Surface Realisation Shared Task (SR'18): Overview and Evaluation Results](https://research.elamur.ai/free/en/papers/mille-2018-first-multilingual-surface-realisation-shared/) — Mille 2018 - [Underspecified Universal Dependency Structures as Inputs for Multilingual Surface Realisation](https://research.elamur.ai/free/en/papers/mille-2018-underspecified-universal-dependency-structures-as/) — Mille 2018 - [The Second Multilingual Surface Realisation Shared Task (SR'19): Overview and Evaluation Results](https://research.elamur.ai/free/en/papers/mille-2019-second-multilingual-surface-realisation-shared/) — Mille 2019 - [BinLin: A Simple Method of Dependency Tree Linearization](https://research.elamur.ai/free/en/papers/puzikov-2018-binlin-simple-method-dependency-tree/) — Puzikov 2018 - [Generating High-Quality Surface Realizations Using Data Augmentation and Factored Sequence Models](https://research.elamur.ai/free/en/papers/elder-2018-generating-high-quality-surface-realizations/) — Elder 2018 - [Head-First Linearization with Tree-Structured Representation](https://research.elamur.ai/free/en/papers/yu-2019-head-first-linearization-tree-structured/) — Yu 2019 - [IMSurReal: IMS at the Surface Realization Shared Task 2019](https://research.elamur.ai/free/en/papers/yu-2019-imsurreal-ims-surface-realization-shared/) — Yu 2019 - [IMSurReal Too: IMS in the Surface Realization Shared Task 2020](https://research.elamur.ai/free/en/papers/yu-2020-imsurreal-too-ims-surface-realization/) — Yu 2020 118 works. Every linked title leads to its card on [Elamur Research](https://research.elamur.ai/free/en/projects/mova/); a «measured» mark leads to what Mova measured with it, on [the measured page](https://md.mova.elamur.ai/free/en/docs/papers/measured/); «to add» marks a work whose card is not in the library yet.