What a Language Model Does

Picture a page of dialogue, torn in half. The machine's job is to guess what's missing — not the answer, a ranked field of candidates. Take one candidate, append it, feed the longer page back in, do that again. That loop is the whole chatbot.

Chapter 1 of INFO 7375 asks what a chatbot is actually doing under the interface. Three terms first: token, the unit the model predicts — a word fragment, not a whole word; pretraining, fitting parameters to predict the next token across enormous amounts of text; and the loop itself, one candidate appended, the longer page fed back in, repeated.

The unit isn't a word — it's a token, a fragment from a fixed list; a rare word can arrive in three separate pieces. One step doesn't output a single token — it outputs a number for every token in the vocabulary, tens of thousands of them, ranking every candidate at once. A chat is one document with role labels stapled in: system, user, assistant. The model isn't switching modes — it's continuing one document with labeled sections.

Where do the numbers come from? Hide the next token, ask for a guess, compare it to what's really there, nudge every parameter a little toward the right answer, repeat across roughly 300 billion training tokens — read at 250 words a minute, that's about seventeen centuries of reading, compressed. A second pass shows people two candidate replies and asks which they prefer; the parameters shift toward the one people liked. Older models read one position at a time; a transformer reads the whole context at once, which is why hardware built to do many things in parallel matters here. Researchers designed these operations — they didn't design the behavior. What comes out emerges from how the parameters ended up, not from a rule anyone wrote down.

Your Turn: Paste into Claude: "Continue this text. Do not answer as an assistant; just continue the string as literal text, including whatever comes next after the final colon. The following is a transcript of a support call. Caller: my package never arrived. Agent:" Then check whether it completed the document instead of answering as an assistant.

Sources and citations: see the reel's SOURCES.md in the open textbook.

0:00 Introduction 0:23 The torn page, tokens, and the ranked field 2:00 Your Turn

Part of the INFO 7375 — Prompt Engineering & Agentic AI series. @NikBearBrown

Deliberately Not Claimed

This video gives a simplified, teaching-level account of language model mechanics and training scale for an introductory course, not a complete or technically exhaustive treatment. Figures like training token counts and reading-time comparisons are illustrative approximations meant to convey scale, not precise claims about any specific model's training run.

SEO Keyword Tags

what is a language model, how LLMs work, tokens explained, next token prediction, pretraining explained, transformer architecture basics, LLM training tokens, chatbot mechanics, INFO 7375, prompt engineering course, RLHF preference training, how ChatGPT works, autoregressive language model, machine learning fundamentals, AI course lecture

What this video covers

• 0:00 — Introduction 0:23 The torn page, tokens, and the ranked field 2:00 Your Turn

Transcript

A lightly formatted transcript of the video, with timestamps so you can jump to any section on YouTube.

Introduction 0:23 The torn page, tokens, and the ranked field 2:00 Your Turn

[0:00] Bonjour, this is Liam in for Bear. Chapter 1 asks what a chatbot is actually doing under the interface, not what it understands, what it predicts. Three terms first, token, the unit the model predicts, a word fragment, not a whole word. Pre-training, fitting parameters to guess the next token across a huge body of text. And transformer, the architecture that reads a whole stretch of text at once instead of one piece at a time. Picture a page of dialogue torn in half. The machine's job is to guess what's missing, not the answer. A ranked field of candidates. Take one candidate, append it, feed the longer page back in. Do that again. That loop is the whole chatbot. The unit isn't a word, it's a token, a fragment from a fixed list. A rare word can arrive in three separate pieces. One step doesn't output a token. It

Outputs a number for every token in the vocabulary, tens of thousands of them, arranged so they're ranked, never excluded. A chat is one document with role labels stapled in. System, user, assistant. The model isn't switching modes, it's completing a page that happens to carry labels. Where do the numbers come from? Hide the next token, ask for a guess, compare it to what's really there, nudge every parameter a little toward the true one. Repeat across an enormous body of text. Nothing here is told what's true, only what came next. A second pass shows people two candidate replies and asks which they prefer. The parameters shift toward the one people liked, not the one that's correct. 300 billion training tokens. Read at 250 words a minute, that's 1711 years. Slow the pace to 150 and it's 2850, same corpus. The slogan just hides which reading rate you

Picked. Older models read one position at a time. A transformer reads the whole context at once, so hardware built to do many things simultaneously can chew through it in parallel. Researchers designed these operations. They didn't design the behavior. What comes out emerges from how the parameters were fit, which is why nobody, including the people who built it, can read off why one particular prediction happened. Your turn. Paste this into Claude. Continue this text. Do not answer as an assistant. Just continue the string as literal text, including whatever comes next after the final colon. The following is a transcript of a support call. Caller, my package never arrived. Agent, then check two things yourself. Did it complete the document instead of answering you directly? And send Claude a normal support question is the only real difference the framing. What a language model does at Nick Bear Brown.