Elamur: **Mova** · [Parse](https://mova.elamur.ai/free/en/parse/) · [Dictionary](https://md.dictionary.elamur.ai/free/en/) · [Translate](https://md.translate.elamur.ai/free/en/) · [Research](https://md.research.elamur.ai/free/en/) · [MCP](https://mcp.elamur.ai/) · [Groupants](https://md.groupants.elamur.ai/free/en/)

[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/tools/elephant/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [tools](https://md.mova.elamur.ai/free/en/docs/tools/) / Elephant — mova-elephant

# Elephant — `mova-elephant`

🐘 The Elephant is one of Mova's animals (Rust on Mova): the keeper of the word graph ([`wordgraph`](https://md.mova.elamur.ai/free/en/docs/wordgraph/)) and of the approach nodes over papers ([`wordgraph/tamp`](https://md.mova.elamur.ai/free/en/docs/wordgraph/tamp/)). See [the animals](https://md.mova.elamur.ai/free/en/docs/repo-docs/animals/).

It keeps the graph **incrementally**, because its program keeps changing: new passes, new parts of a running pass and new papers are added without a full rebuild, and a changed program recounts only the parts it affects. It reads English with Mova, works in a standing loop under Crab, and asks two consultant terminals only what the counts cannot decide; their answers are review-only proposals, never graph writes.

## Commands

```
elephant init STORE [K K_CORE MIN_COUNT MIN_OOV]   a new, empty store (default 32 128 3 20)
elephant add STORE PASS_DIR [PART…]               count the finished parts of a pass (all, or those named)
elephant retract STORE PASS/PART                  take a counted part out (subtracted at the next tamp)
elephant tamp STORE [FAN_IN [major]]              merge the FAN_IN smallest units (8), or all with major
elephant view STORE                               build the graph of what is counted (reused when unchanged)
elephant status STORE                             the store as JSON
elephant compare A.wg B.wg                        the same graph, every section but provenance?
elephant lists STORE WRONG_TSV RIGHT_TSV          keep a snapshot of a version of the hand lists
elephant plan STORE KEY_JSON                      what a new program key means for the counted parts
elephant reprogram STORE KEY_JSON PLAN_JSON       attest, retract and rekey by a plan
elephant suspects STORE [N]                       suspicious edges of the view (nothing is asked)
elephant ask CONFIG DEPUTY_CONFIG ID < text       read English, answer from the graph or the deputy
elephant serve CONFIG DEPUTY_CONFIG SOCKET LOCK   answer English on a private socket (one per LOCK)
elephant client SOCKET ID [urgent] < text         send English to a serving Elephant
elephant work CONFIG [CONSULTANTS|- [STEPS|- [SECS]]]   the standing loop (SECS 300)
elephant work-summary CONFIG                      events of the loop
```

Build and test:

```
cargo build --release --manifest-path tools/elephant/Cargo.toml --offline --locked
MOVA_OCTOPUS_TEST_MODEL=$MOVA_DATA/…/en.bin cargo test --release --manifest-path tools/elephant/Cargo.toml --offline --locked
```

Store commands that write take the store's writer lock (`STORE/writer.lock`); a running loop holds it.

## The incremental graph

**Model.** The graph is a function of the multiset of pass parts it counts, `G = build(Σ parts)` (`wordgraph::compile::build`). PMI needs the global marginals (`N_r`, `c(h,r,·)`, `c(·,r,d)`), and any new event of a relation changes `N_r` and so every PMI of that relation: no edge of `G` can be updated locally. What is incremental is the counting. Counts form a commutative group under addition, so the merge of parts can be done in any order and grouping, and a part can be subtracted exactly.

The store is a log-structured merge (LSM) store of counts with the graph as its index:

- **runs**: immutable canonical merges of parts, written by the Elephant (`src/merge.rs`): the summed counts restricted to positive ones, in key order, with exactly the out-of-dictionary words their records use, in the part format, so the `wordgraph` build reads a run like any part;
- **deltas**: parts added since, read in place from their pass directories (never copied);
- **retractions**: parts inside a run to subtract at the next tamp (a recount replaces an old part);
- **the view**: the graph built from runs + deltas. Queries read it: one `wordgraph::Graph` file, memory-mapped, `Clone + Send + Sync`, the same API as any graph.

**Operations.** `add` hashes the part, checks its key and appends a delta (O(part bytes)). `tamp` merges the `fan_in` smallest units (runs and deltas) into one run — size-tiered compaction, so a record is rewritten O(log_fan_in N) times over the store's life and a view build reads a bounded number of inputs; `major` merges everything. `view` stages the runs and deltas as one pass of symbolic links and builds the graph; it is named by its **lineage** (the hash of the sorted part hashes, the key and the parameters) and reused while the lineage is unchanged, also across tamps.

**Invariants.**

- I1, counted once: a part's sha256 occurs in at most one run or delta; a part with a counted hash is skipped, and a part with a counted name and other contents is refused (retract the old one first).
- I2, one program: every counted part has the store's key, or an attestation for it (below).
- I3, exactness: the run of any merge tree over parts P equals the run of merging P at once (byte for byte), so a view over runs + deltas equals the `wordgraph` build over the original parts in every section but provenance, given the same OOV registry state (registry ids are append-only: a word keeps the id it got first).
- I4, one graph per lineage.

Every change writes a new generation: `history/<generation>.json`, then `manifest.json` replaced atomically, then a line in `journal.jsonl`. Nothing is deleted; superseded runs are listed in the manifest as reclaimable.

**Reprogramming.** A part's key is what decides its records: part format, extraction rules version, the fingerprint of the hand lists it was extracted with, the parse models, language and dictionary fingerprint. `plan` sorts every counted part under a new key into *ok*, *attest* and *recount*. Another rules version, model or format: recount. Only the hand lists changed: only `compound_reading` entries change extraction (the other kinds act at query time). A new entry (h, r, d) suppresses only emissions of the triple (h, r, d) itself — and the role swaps it would take part in, which exist only if it is emitted — so a part holding no record of (h, r, d) under any UPOS is provably unchanged; a removed entry changes a part only where it suppressed something, which the part's sidecar counts. Those parts are attested; the rest are retracted and come back recounted in a new pass. `reprogram` applies the plan and adopts the key. The list versions are kept as snapshots in the store (`lists/<fingerprint>/`).

**Cost model.** Add: one hash per part. Tamp: two streaming reads of its inputs' records (O(R log k)), two of their OOV tables, one write; memory 4 bytes per input OOV entry plus 9 bytes per word of their union, no strings or records held. View: one `wordgraph` build over the staged inputs. Prospective prediction (2026-11-06, before the first real run): on the 50 000-article pass (100 parts, 103 M events) a tamp of 8 parts in seconds, a view over a few runs plus deltas about as long as the full build (≈ 1 min), memory below the full build's.

## Measurements (2026-11-06)

Pass `wiki-t1-000-100-r2` (50 000 Wikipedia tier-1 articles, 100 parts, 103.1 M events, extraction rules 2, part format 1), `k = 32`, `k_core = 128`, `min_count = 3`, `min_oov = 20`; a shared box (load 11–15 on 20 cores), heavy steps under `tools/jobs` with a 4 GB limit.

| step | wall | peak RAM |
| --- | --- | --- |
| full `wordgraph build` of the 100 parts (reference) | 54 s | 1.25 GB |
| `add` 95 parts (hash each) | 1.0 s | — |
| 14 tamps with fan-in 8 (100 deltas → 8 units), then a major tamp | 99 s in all, up to 23 s each | ≤ 0.05 GB |
| `add` the last 5 parts (another store, 4 runs + 5 deltas) | 0.03 s | — |
| `view` over 4 runs + 5 deltas | 39 s | 0.73 GB |
| major tamp (→ 1 run, 35.1 M records, 143 MB) | 21–23 s | 0.04 GB |
| `view` after the major tamp | reused (same lineage) | — |

- The view over runs + deltas equals the full build (`elephant compare`: every section but provenance).
- Three stores with fan-in 8, 4 and 8 (two merge trees, two versions of the merge's memory layout) wrote the same final run, byte for byte (SHA-256 `acad1f59…`).
- English through `serve` on that view: 107–159 ms a question (client process, graph open, parse, answer); the `ask` command loads the model per call (≈ 2 s).
- `suspects` on that view: 7 977 suspicious edges (5 190 compound shadows) in 0.36 s, identical on two runs. Of the 23 edges of the hand list of the wrong (`compound_reading`, branch `word-graph-all`) present in this graph, 10 are among them (ranks 44, 101, 112, 735, 1612, 1833, 2495, 2554, 2988, 4233); the class-outlier signal alone found 1 of 23. The top shadows are mostly participial compounds («supporting actor», «backing vocals», «highest-grossing film», «recurring role», «defending champion», «failed attempt», «best-selling album») next to real pairs («speak a language», «increase the risk»): telling them apart is the consultants' question.

The predictions held for the view (39 s against ≈ 1 min) and the memory; the tamps of the larger runs took longer than «seconds» (up to 23 s).

**The whole-Wikipedia pass** (`wiki-all-v3`, extraction rules 3, part format 2, running): its first 11 parts (≈ 223 000 articles, 245.7 M events, 126.0 M records) were added in 0.8 s and tamped into one run of 77.8 M records, 2.0 M OOV words and 300 MB in 69 s at 0.10 GB. Built with `wordgraph` of branch `word-graph-all` (which reads format 2), the view from that run (54 s, 0.65 GB) equals the view from the 11 original parts (60 s, 0.68 GB). Prospective prediction for the finished pass (about 30 times as many articles): a major tamp of 30–60 min under 2 GB, a run of 3–6 GB; the view's memory is the `wordgraph` build's, not the Elephant's.

## In English

A prompt is read first by the **question reader** (`src/ask.rs`) over Mova's own parse (`world::tree::Tree::parse_cased` on the `en` annotator), with a small table of frames, each one graph query:

| frame | example | query |
| --- | --- | --- |
| `does` | What does a dog do? What can a dog do? | verbs whose subject is the noun |
| `objects` | What does a cat eat? | objects of the verb; the subject checked |
| `subjects` | Who elects a president? What barks? | subjects of the verb; the object checked |
| `isa` | What is a dog? | classes of the noun |
| `is-a` | Is a dog an animal? | the class edge checked |
| `plausible` | Is 'the food fruits the man' plausible? Does “the dog bites the man” make sense? | the quoted sentence parsed; each subject, object and copular role scored with the isa back-off; the roles swapped |

The deputy's capabilities are a closed vocabulary (typed slots take URLs, hosts, identifiers and quoted text), so a question about arbitrary words cannot be a capability; the frame table plays that role with the same native parse. What the reader does not take — statements, negations, several sentences, unlisted frames — goes to Mova's native deputy with the Elephant's hands (`deputy.example.json`, handler `elephant`): `status` and `proposals` observe ("How many words do you know?", "How big is the graph?", "Is the graph tamped?", "How many proposals do you have?"); `tamp` and `refresh` act ("Tamp the graph.", "Please refresh the graph.") by leaving a request for the standing loop — the English service never writes the store. "Delete the graph." and "Do not tamp the graph." are refused with a review candidate.

Answers on the 50 000-article view (2026-11-06): «In my graph a dog most often goes, eats, barks, bites, skips and dies.» «Those who most typically elect in my graph: Assembly, voter, member, council, constituency and Congress. «elect a president» is in my graph, seen about 470 times.» «I find «the food fruits the man» unlikely. I have no evidence for «food» as the subject of «fruit». …» Absence is weak evidence and the answer says so; «the dog bites the man» comes out unlikely and «the man bites the dog» plausible on this graph (no «bite → man» object edge was kept, while «man» as a subject of «bite» and «dog» as its object were), which is a fact about these counts, not about dogs.

## Another language

An Elephant keeps one language: `language` in its configuration (`en` by default, or `fr`), and a store counts parts of one key, whose language is part of it; the loop stops when the store's language is not the configuration's. The questions stay English and are read with the English `en.bin`. A word of the graph's language is quoted: a quoted single word is read in place of an English stand-in (a noun phrase, else a verb), so «What does "chien" do?» is the `does` frame on «chien», «What is "Paris"?» the `isa` frame, «Who "élire"?» the `subjects` frame. Answers about a non-English graph say «my French graph» and quote its lemmas, without English articles or inflection; an English word is not a word of a French graph («I do not know the word «dog» in my French graph»). The consultants' question names the graph's language and gives a French example of a compound reading.

Measured 2026-11-06 on a store of the top 1 000 French articles (one part, 3.67 M events, 12 367 words): «In my French graph «roi» is most often the subject of «mourir», «acheter», «accorder», «construire», «accepter» and «décider».»; «In my French graph «Paris» is most often «ville», «capitale» or «siège».»; «…«guerre» is most often the subject of «éclater», «opposer», «reprendre» …». Quoted words must be lemmas (there is no French lemmatiser in the reader). `tests/english.rs` checks these frames on a synthetic French store.

## Consultants

`consultants.example.json`: two profiles (Sol through the Codex CLI in our deployment), each with its own `mova-terminal` (`terminal-elephant-{1,2}.example.json`, held by `crab-tmux-worker` with the consultant prompt [tools/terminal/prompts/animal-terminal.md](https://md.mova.elamur.ai/free/en/docs/tools/terminal/prompts/animal-terminal/)), purposes `edges` and `approaches`. The loop asks one bounded question at a time (at most 10 edges or 3 approach nodes, under 24 000 bytes), round-robin over the terminals whose pacing (600 s) passed, alternating the purposes:

- **edges** (`src/suspect.rs`): is a suspicious edge a parse error, a compound reading, figurative, real or unsure? Two graph-only signals choose them: the *compound shadow* — a verb–noun core edge whose noun is also modified by the verb itself (`amod`: «eating disorder» counted as eat + object disorder) — and the *class outlier* — a strong dependent that shares no `isa` class with the other typical dependents of its head.
- **approaches**: do the members of an approach node (`elephant-tamp build`, `nodes.jsonl`) explain the same approach? Lowest mean fit first.

Rules (as the teachers of Nautilus and Sea Turtle): no question without `operator_word` in the configuration; the terminal must be the profile's own; every question is journaled before and after with its hash and size; a question whose hash is in the journal is never asked again, and an uncertain one is never replayed. Answers are parsed strictly (one line per item, fixed columns; a compound reading must end with one of the edge's words) and kept in `WORK/proposals/proposals.jsonl` with the question's and the answer's hashes, the hand-list kinds also as `wrong.proposed.tsv` in the lists' six columns. Nothing is admitted: a person moves a reviewed line into `wordgraph/seeds/wrong.tsv` or `right.tsv`, which is a reprogramming (above).

## The standing loop

`work` is the store's only writer. One step: take the English service's requests; add up to `add_per_step` finished parts of the configured passes; tamp when more than `fan_in` units wait (or a retraction, or a request); rebuild the view when the events outside it reach `view_share` of it (at most once per `view_min_secs`, or on request); when papers are configured and at least `min_new` are new, run `elephant-tamp read` (cached per text: only new papers are read) and `elephant-tamp build` into a new folder; ask one consultant question when one is due. Each step is one line of `WORK/work.jsonl`; an error stops the loop (fail fast, nothing retried); `STOP` in the work folder ends it.

`crab-workers.example.json` lists the English service, the loop and the two terminal holders, all `autostart: false`.

## Tests

`tests/lsm.rs`: a store fed one pass, tamped in pairs, then another pass as deltas equals the full build; a major tamp reuses the view and its rebuild is equal again; another order, merge tree and fan-in give the same run bytes; a retraction equals the full build without that part; reprogramming on a list change attests the parts without the new compound's edge and recounts only the one that holds it. `tests/work.rs`: the loop adds, tamps and rebuilds, takes a request once, finds the planted suspicious edge, holds the question without the operator's word, keeps the answer as one review-only proposal and leaves the graph unchanged. `tests/english.rs` (needs the model): the frames on a synthetic graph, the deputy's status, proposals, tamp and refresh, and the refusals. Unit tests: part formats, signed merges, keys and list changes, consultant rules, strict answer parsing, quotations.

Negative controls (each red, restored, 2026-11-06): the comparison itself (a graph missing a pass differs); a part under another rules version, a file that does not hash to its sidecar, a retraction of what is not counted, a changed part at tamp time — refused; an entry on an edge every part holds attests nothing; merging by `max` instead of sum (the equality test red); the counted-hash skip removed (double counting, red); the negation check of the reader removed (the English tests red).

## Sources

Library ids are folders of the paper library; works marked «outside» have no card yet (`docs/usage.tsv`).

- LSM store with size-tiered compaction: O'Neil et al. 1996 (outside); Bigtable, Chang et al. 2006 (outside); a GPU LSM dictionary, ashkiani-2017. Sorted runs merged k-way with a heap: Knuth 1998 vol. 3 (outside).
- Signed counts as a commutative group (retraction = weight −1): no card; the invariants I1–I4 are this crate's own.
- Part compression: zstd, Collet & Kucherawy 2021, RFC 8878 (outside).
- Scores behind the graph: see [wordgraph — the word graph](https://md.mova.elamur.ai/free/en/docs/wordgraph/) (church-1990, dunning-1993, evert-2005).
- Class-outlier suspects: resnik-1993, rooth-1999.

## Limits

- The view is exact but not free: after new parts it is rebuilt by one `wordgraph` build over runs + deltas (no edge is updated in place, since PMI is global). Between rebuilds queries read the last view; the status says when it lags.
- Retraction needs the original part file as counted (same hash). Superseded runs are not deleted by the Elephant.
- The store keeps one part format per store: `wordgraph` on `main` reads format-1 parts only; the whole-Wikipedia pass writes format 2 (branch `word-graph-all`), which this crate reads and merges, and whose build reads it.
- `wordgraph compact` (branch `word-graph-all`) tamps a pass's consecutive parts once before one build; the Elephant's runs are the standing form of the same idea (retraction, lineage, program keys, any merge tree). Both write parts the build reads.
- Approach nodes are rebuilt whole (≈ 1 min for 1 763 papers); only the reading of papers is incremental.
- The question frames are a fixed table; what they do not cover goes to the deputy and is refused there.
- The suspicious-edge signals choose what to ask; they are not judgements. Their precision is unmeasured until reviewed answers exist.
- Another language: the quoted words must be lemmas; the plausibility frame parses its quoted sentence as English.

## Sources

- The graph's scores and lists: [`wordgraph`](https://md.mova.elamur.ai/free/en/docs/wordgraph/#sources) (evert-2005 and the works listed there).
- Log-structured merging (size-tiered compaction) as in LSM stores: O'Neil et al. 1996 (no card).
- The French graph: UD French GSD (guillaume-2019) through the French pass, French Wikipedia (CC BY-SA 4.0, internal).
- [← previous Mova Eel](https://md.mova.elamur.ai/free/en/docs/tools/eel/)
- [next → mova-enrich — grounded corpus enrichment and bounded self-study](https://md.mova.elamur.ai/free/en/docs/tools/enrich/)
- [↑ back to tools](https://md.mova.elamur.ai/free/en/docs/tools/)