--- title: How a language model learns: guessing the next word — a lesson of Mova's school · trees and pragmatics description: «How a language model learns: guessing the next word», a lesson of the ai course of Mova's primary school, sentence by sentence with dependency trees and pragmatics. It begins: A chatbot is built on a "large language model": large because it is enormous, language because it works with words, model b --- [Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/) [home](https://md.mova.elamur.ai/free/en/) / [corpora](https://md.mova.elamur.ai/free/en/corpora/) / [school](https://md.mova.elamur.ai/free/en/corpora/school/) / [AI](https://md.mova.elamur.ai/free/en/corpora/school/ai/) / How a language model learns: guessing… # How a language model learns: guessing the next word Lesson 2 of the ai course of Mova's primary school: 17 sentences, each with its syntax tree and pragmatics. The lessons are written for Mova, and their agreed trees are part of Mova corpus 0.4 (CC BY-SA 4.0). How Mova does on its tests: [the lesson at school](https://md.mova.elamur.ai/free/en/school-run/ai/2-guessing-the-next-word/). - [0](https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/0/) — A chatbot is built on a "large language model": large because it is enormous, language because it works with words, model because it is a copy of the patterns in how people write. - [1](https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/1/) — It learned by playing a giant game of "guess the missing word" over and over with more text than you could read in thousands of lifetimes. - [2](https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/2/) — Now, when you ask something, it builds an answer a little piece at a time, each time picking a word that fits well after the words before it. - [3](https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/3/) — This is why it can say "I understand how you feel" in a kind voice: kind people often write that. - [4](https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/4/) — Whether that counts as real understanding is something scientists still argue about, but it has never felt sad in a body. - [5](https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/5/) — The answer is worked out on huge computers far away, in buildings called data centres; nobody is typing it. - [6](https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/6/) — "Large language model" means a huge program that has learned the patterns of how words are used. - [7](https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/7/) — A language model learned by predicting missing or next words in enormous amounts of text. - [8](https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/8/) — It writes an answer piece by piece, each time choosing a likely next word. - [9](https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/9/) — Its training text stops at a certain date, so a bot without a clock or web search may not know today's time or news. - [10](https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/10/) — When the program is switched off, nothing happens inside it: it does not sleep, rest or dream. - [11](https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/11/) — A bot has no body, so it is neither a boy nor a girl; a name and voice are design choices. - [12](https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/12/) — Bots often give the most common human answer, so many of them pick 7 when asked for a number from 1 to 10. - [13](https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/13/) — The text a bot learned from had good and bad parts; companies filter it and train the bot to be polite, but they do not catch everything. - [14](https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/14/) — A bot is more than a parrot, because it can mix ideas in new ways, but everything it knows came from what people made. - [15](https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/15/) — Picture-making AI learned from millions of pictures with descriptions and builds new pictures that fit the words. - [16](https://md.mova.elamur.ai/free/en/corpora/school/ai/2-guessing-the-next-word/16/) — Chatbots run on giant computers in data centres; the words appearing one by one is just how the answer is shown. - ← previous: [What is AI? Pattern-guessers, not people](https://md.mova.elamur.ai/free/en/corpora/school/ai/1-what-is-ai/) - next →: [Sounding right and being right](https://md.mova.elamur.ai/free/en/corpora/school/ai/3-sounding-right-and-being-right/) - ↑ back to [AI](https://md.mova.elamur.ai/free/en/corpora/school/ai/)