Study Guide

How Language Models Write

How an AI chat assistant writes a reply: by predicting how likely each possible next piece of text is, choosing one and repeating. What tokens are, where the probabilities come from, why fluent text can still be wrong, and what the temperature setting changes.

Lesson 8.2 AI Foundations

For: Learners aged 14 and over (Key Stage 4, sixth form) and adult beginners; no programming or mathematics beyond percentages

When you type a question into an artificial intelligence (AI) chat assistant, the reply appears a few words at a time, as if someone were typing it. This lesson explains what is happening behind that reply: how the program decides what to write next, why the same question can get a different answer each time, and why an answer can read perfectly well and still be wrong.

People now use AI assistants to draft emails, summarise documents and answer questions at work. Knowing how the text is produced tells you when to rely on it and when to check it. That habit matters in security work too, where a confident but wrong answer about an alert or a log entry can lead to a bad decision.

Stage: Think

Finish the sentence

Start with something you already do every day, before any technical words. Read this unfinished sentence.

The teacher asked the pupils to open their …

Think it through

What one word would you put next, and why that word?

Show what happens

Most people say books. Some say laptops or notebooks; almost nobody says umbrellas.

Notice two things about what you just did. You did not look the answer up: you used patterns from years of reading and hearing sentences like this one. And you had more than one candidate. Several words came to mind, and some felt much more likely than others.

Stage: Predict

More than one possible next word

Think it through

If 100 different people finished the same sentence, what would you expect?
Show the answer

Answer: B. Most would write one of a few common words; a handful, something unusual.

If you counted the answers, you could turn them into percentages. Perhaps 52 people write books, 22 write laptops, 14 write notebooks, and the last 12 are spread across rarer words.

That list is the key idea of this whole lesson: a set of possible next words, each with a probability. A probability is how likely something is, written here as a percentage. The probabilities of all the possible next words add up to 100%, because one of them must come next.

Here are the counts above as probabilities, with three rarer words added:

Next wordProbability
books52%
laptops22%
notebooks14%
minds6%
windows4%
umbrellas2%

Stage: Experiment

A language model makes the same kind of prediction

A language model is a computer program that does what you just did, on a vast scale. Give it some text, and it works out a probability for every possible next piece of text. It does not produce an answer directly: it produces a list of probabilities, like the table above.

To write, the program then repeats four moves:

  1. Predict: work out the probability of every possible next piece of text.
  2. Choose: pick one of them.
  3. Add: put the chosen piece on the end of the text.
  4. Repeat: predict again, from the slightly longer text.

Try being the model. Choose each next word yourself, or let the panel choose, and watch the sentence grow.

Try it: write a sentence one word at a time

Each list shows the probability of every possible next word. Choose one, take the most likely, or pick at random by probability. The sentence ends at a full stop.

Text so far: The teacher asked the pupils to open their

Predictions for the next word, after “their”

  1. books52.0%
  2. laptops22.0%
  3. notebooks14.0%
  4. minds6.0%
  5. windows4.0%
  6. umbrellas2.0%

Always taking the most likely word gives: The teacher asked the pupils to open their books at page ten.

A toy model with invented probabilities, built for this lesson. It looks only at the last word and knows fewer than twenty words. A real language model takes the whole text into account and scores tens of thousands of possible pieces, or more, at every step.

Whatever you chose, you did what a chat assistant does for every reply, however long: predict, choose, add, repeat. The reply is generated a piece at a time. It is not fetched from somewhere as a finished answer.

A phone keyboard’s next-word suggestions work on the same principle, so the comparison is a fair place to start. A language model goes much further. It takes the whole text into account, including your question and everything it has written so far, and it learned its probabilities from vastly more text than any keyboard. That is why it can write whole paragraphs that stay on topic. It is still predicting, one piece at a time, and nothing in that process checks whether what it writes is true.

Stage: Discover

Pieces of words: tokens

So far each piece has been a whole word. That is a simplification, and the reason is worth working out.

Think it through

A model must be able to write any text, including names it has never seen, new slang and spelling mistakes. Which approach would work best?
Show the answer

Answer: A. Build text from reusable pieces: common words whole, rare ones in parts.

Language models work with tokens: small pieces of text. A common word such as books is usually a single token. A longer or rarer word is split into several. As an illustration, unbelievably might become un, believ and ably; real models split words in their own ways. Punctuation marks are tokens too.

So the accurate name for the prediction step is next-token prediction: the model predicts the next token, not the next word.

Two consequences are worth knowing now.

  • Each model splits text in its own way. The same sentence can be a different number of tokens in different models.
  • Limits and costs are counted in tokens. How much text a model can take into account at once, called its context window (the subject of a later lesson), is measured in tokens, and many services charge by the token.

The panels in this lesson treat each word as one token, to keep them simple.

Stage: Explain

Where the probabilities come from

Think it through

The model has almost certainly never seen your exact sentence about the teacher. How can it still give “books” a high probability?
Show the answer

Answer: C. It learned general patterns of language from huge amounts of text.

The model learned patterns. The rest of this step explains how.

Inside, a model is a very large set of numbers, called parameters, together with the calculation that uses them to turn an input into an output. For a language model the input is text and the output is the list of probabilities for the next token.

The parameters start with no useful values. Training gives them their values:

  1. The model is shown a piece of real text, with the next token hidden.
  2. It predicts the probabilities of the next token.
  3. Its prediction is compared with the token that really came next.
  4. Every parameter is nudged slightly, so that the right token would have been a little more likely.

Repeated an enormous number of times, over an enormous amount of text, those small nudges leave the parameters holding the patterns of the language: grammar, common phrases, and a great many facts that appeared often in the text.

A large language model (LLM) is a language model with a very large number of parameters, often billions, trained on a very large amount of text. The chat assistants you may have used are built on them.

Training happens before anyone uses the model, and it is slow and expensive. Using the finished model to produce an output, such as a reply to your question, is called inference. During inference the parameters stay as they are: chatting with a model does not retrain it as you type.

Do not confuse: Training and inference

When?Training
Before the model is used. Its parameters are adjusted, over a huge amount of example text, until its predictions are good.A company spends weeks training a new model.
When?Inference
Every time the finished model is used. Its parameters are not changed; they are used to make a prediction.You ask a question and a reply appears.
Go deeper Training in more detail, and where the simple picture stops
  • Training happens in stages. After training on general text, a model behind a chat assistant is usually trained further, on examples of helpful conversations and on people’s ratings of its answers. That is why it answers questions instead of simply continuing your text. It still writes by predicting one token at a time.
  • Models can memorise. Parameters mostly capture patterns, but a model can sometimes reproduce, word for word, a passage that appeared many times in its training text. That is one reason why what goes into training matters.
  • Models can be updated. A model can be trained again later on newer text, and some systems learn continuously from new data, which is called online learning. That is a deliberate step taken by the people who run the system, not something that happens as you chat.
  • An application is more than its model. A chat application can save details from earlier conversations and add them to what it sends to the model, or search the web and add the results. The application changes what the model is given to read; the model’s parameters stay the same.

Stage: Explain

The model and the app around it

When you use a chat assistant, you are not talking to the model alone. The model is the engine: the parameters and the calculation that predict the next token. The application is everything built around it: the chat window, the instructions it adds before your message, any memory of earlier chats, and sometimes a web search whose results it pastes in for the model to read.

The same model can sit behind several applications, each with its own settings, and the same application can change the model it uses.

Check your understanding

Two apps use the same model. Asked about yesterday’s football results, one gives the right scores and the other says it does not know. What is the most likely reason?
Show the answer

Answer: A. The first app searched the web and gave the model the results to read.

A model only knows what was in its training text and what it is given to read now. Lesson 3 looks at what that means when you need up-to-date answers.

Stage: Discover

Fluent is not the same as true

You now know enough to work out one of the most important facts about AI writing tools. Read this example, then answer the question after it.

Ms Okafor, a head of year, is testing an AI assistant. She asks it about a school she has made up:

Who was the first head teacher of Brindle Vale Academy?

Because she made the school up, the model has learned nothing about it. Here is what happens, step by step.

  1. The model predicts how the reply starts. Replies to “Who was…?” questions very often begin by repeating the question, so tokens such as The first head teacher are likely.
  2. It predicts what follows “…Brindle Vale Academy was”. In real text, that position is almost always filled by a person’s name, often with a title. So name-like tokens such as Dr and Margaret now have high probabilities.
  3. It keeps going. After a name, dates are common: who led the school from 1998 to 2011.
  4. The result: “The first head teacher of Brindle Vale Academy was Dr Margaret Ellison, who led the school from 1998 to 2011.” Every token was a likely continuation, so the sentence reads exactly like a true answer.
  5. Change one fact. Ask instead about a famous school whose first head teacher appears in thousands of pages of text. The same four moves now make the real name the most likely continuation, so the answer is very probably correct.

The process was identical both times. The answer was right only because the patterns the model learned happened to match the facts.

Check your understanding

Why can a language model give a confident, well-written answer that is wrong?
Show the answer

Answer: B. It writes likely-sounding text, and nothing in that checks facts.

The model writes what is likely, and likely is not the same as true. A fluent, confident answer that is false or invented is called a hallucination.

Many assistants are now trained to say when they are unsure, and some search the web or a document collection before they answer. Those measures make hallucinations less frequent. They do not remove them, because the writing is still done by predicting likely tokens.

The practical rule follows directly: treat a fluent answer as a draft, and check names, numbers, dates, quotations and references against a reliable source before you rely on them.

Stage: Predict

Choosing: the most likely, or not?

Every step so far has had a choice in it: the model gives probabilities, and then one token has to be picked. How should it pick?

Think it through

Imagine a model that always picks the single most likely next token. What would you expect from it?
Show the answer

Answer: A. The same reply to the same prompt every time, often plain or repetitive.

Always taking the most likely token is predictable. Research on text generation found that it also tends to produce dull text that keeps repeating itself.

So most AI writing tools use sampling: they pick the next token at random, but in proportion to its probability. A token with a probability of 52% is picked about half the time; one with 2% is picked now and then. You did this with the button “Pick at random, by probability”.

Sampling is the main reason why the same question can get different answers. Each reply is one of many likely paths through the probabilities.

Stage: Experiment

Temperature: how adventurous the choice is

Temperature is a setting that changes how the probabilities are used when a token is sampled. It does not change what the model has learned. It reshapes the probabilities just before the choice is made.

Think it through

Before you move the slider below: what do you think happens to the probability of “umbrellas”, the least likely word, when the temperature goes up from 1.0 to 2.0?
Show the answer

Answer: B. It goes up, because higher temperatures even the probabilities out.

Raising the temperature evens the probabilities out, so unlikely words gain and likely words lose.

Try it: change the temperature

The six bars are the probabilities of the next word. Move the slider, and use the button to sample twenty words at the temperature you have chosen.

Text so far: The teacher asked the pupils to open their …

Probability of each next word at temperature 1.0

  1. books52.0%
  2. laptops22.0%
  3. notebooks14.0%
  4. minds6.0%
  5. windows4.0%
  6. umbrellas2.0%

At temperature 1.0 the probabilities are the model’s own: “books” is the most likely word, but not the only one.

Next wordTemperature 0.2Temperature 1.0Temperature 2.0
books98.5%52.0%33.5%
laptops1.3%22.0%21.8%
notebooks0.1%14.0%17.4%
minds<0.1%6.0%11.4%
windows<0.1%4.0%9.3%
umbrellas<0.1%2.0%6.6%

Invented probabilities for six words, built for this lesson; at temperature 1.0 they are the same as in the first panel. The arithmetic is the real one. Services use their own temperature ranges, and often combine temperature with other settings that also limit the choice.

What the slider shows:

  • Below 1, the differences between probabilities are exaggerated. The likely words become even more likely, and at very low temperatures the model almost always picks its top word, so the output is consistent and predictable.
  • At 1, the model’s own probabilities are used unchanged.
  • Above 1, the differences shrink. Unlikely words get a bigger share, so the output is more varied. At high values it can wander off topic or stop making sense.

A low temperature suits tasks where consistency matters, such as pulling the same facts out of a document every time. A higher one suits brainstorming, where variety is the point. Not every service lets you change it.

Go deeper The arithmetic behind the slider

Inside the model, each possible token has a score. The scores are turned into probabilities by a calculation called the softmax, which makes every probability positive and makes them add up to 100%. Temperature divides every score by the temperature before that calculation.

That turns out to be the same as raising each probability to the power 1 ÷ temperature and then rescaling so that they add up to 100% again. At temperature 0.5 the power is 2:

Next wordAt 1.0SquaredAt 0.5
books52%0.270478.6%
laptops22%0.048414.1%
notebooks14%0.01965.7%
minds6%0.00361.0%
windows4%0.00160.5%
umbrellas2%0.00040.1%

The squares add up to 0.344, and each percentage in the last column is its square divided by 0.344. Squaring hits small numbers hardest, which is why the rare words almost vanish. A temperature above 1 means a power below 1, which pulls small numbers up instead.

Nothing can be divided by a temperature of exactly 0, so services that accept 0 treat it as “always take the most likely token”.

Stage: Check

Check your understanding

Answer these without scrolling back if you can. If you get one wrong, the feedback will point you in the right direction.

Check your understanding

A colleague says: “Set the temperature as low as it will go and the assistant will stop making mistakes.” What is the best response?
Show the answer

Answer: C. Disagree: it makes answers more predictable, not more accurate.

Temperature changes how predictable the choice is. It cannot make a wrong but likely answer true.

Check your understanding

Which statement about tokens is accurate?
Show the answer

Answer: B. A rare word may be split into several tokens.

Tokens are pieces of text. Many common words are one token each, rarer words are split into several, punctuation counts, and each model splits text in its own way.

Check your understanding

A school uses an AI assistant to draft letters to parents. Which of these is inference?
Show the answer

Answer: C. The assistant drafts a letter from the office manager’s request.

Inference is using the trained model to produce an output. Adjusting the parameters is training, and collecting text happens before training.

Stage: Apply

Apply it in a new situation

These questions use the same ideas in situations the lesson has not shown. For the strongest test of your understanding, come back to them on another day and answer each one at the first attempt.

Apply it

Two members of staff ask the same AI assistant exactly the same question about the school’s data retention policy, a minute apart, and get differently worded answers. What is the most likely explanation?
Show the answer

Answer: B. Each next token is sampled, so one prompt can lead to different text.

Different wording from the same prompt is normal: the next token is sampled at each step. It is not a fault, and it does not mean either answer is correct. Both should be checked against the policy itself.

Apply it

An IT technician asks an AI assistant to summarise a long security incident report. The summary says the incident began on 14 March. What should the technician do before putting that date in an email to the head teacher?
Show the answer

Answer: C. Check the date against the original report, and correct it if needed.

A summary is generated token by token, so a date in it can be wrong even when the report is right there in the prompt. Before a detail is used for a decision, check it against the original.

Stage: Consolidate

What to remember

Think it through

Without looking back, explain to someone who missed this lesson how a chat assistant writes a reply, in no more than three sentences.

Show a model answer

It predicts how likely each possible next token is, from patterns learnt in training; it chooses one, usually by sampling; it adds it to the text and repeats until the reply is finished. Nothing in that loop checks whether the reply is true.

Check your understanding

In predict, choose, add, repeat, which move is the main reason one question can get differently worded answers?
Show the answer

Answer: A. Choose: the next token is sampled, so different likely tokens can be picked.

The probabilities come from the trained parameters and stay the same for the same text; the choice between them is where chance, and so variety, comes in.

  • A language model predicts. For any text, it works out how likely each possible next token is.
  • Text is generated by repetition: predict, choose, add, repeat. Nothing is fetched as a finished answer.
  • Tokens are pieces of text, often whole common words, sometimes parts of words, and each model splits text its own way.
  • Training sets the parameters; inference uses them. Using a model does not normally change it.
  • Fluent is not the same as true. Nothing in the process checks facts, so check anything that matters.
  • Sampling makes answers vary, and temperature sets how much. A low temperature makes answers predictable, not accurate.

Common misunderstandings

A language model writes its answer by prediction. Some applications add a search step, but the text itself is still generated, and can still misreport what the search found.

Confidence is a matter of style, and style is exactly what a language model is good at predicting. It says nothing about accuracy.

A low temperature removes variety, not error. A wrong answer that is likely becomes a wrong answer that is repeated.

During inference the parameters do not change. An application may store your conversation and reuse it, but that is the application, not the model, learning.

Lesson 3 builds on this one: how much text a model can take into account at once, and why a model knows nothing after a certain date.

Sources and scope

  • What this lesson is. The second of four introductory AI Foundations lessons, and a draft awaiting the site owner’s review. It began as a draft written for the Cybersecurity Study Guide (“How Language Models Generate Text”, 26 October 2026), brought here with one added step, “The model and the app around it”, and new links to the Computing curricula. It is an introduction, not preparation for any certificate. Its curriculum links are provisional: the wording of each linked specification point has been checked against the published specification, but no link has yet been verified by the site owner. The links show where the lesson may help with a course; they do not imply that any examination board has reviewed or endorsed it.
  • Planning reference. Its scope was planned against areas 2.1, 2.2, 2.3, 2.5 and 6.1 of the UK AI Council’s AI Foundation technical syllabus (version 1.0), using the site’s own mapping. Nothing here is endorsed by, affiliated with or approved by the UK AI Council, and no text or example is taken from its course materials.
  • Independently checked: the definitions against Google’s Machine Learning Glossary; the temperature arithmetic (the softmax with a temperature) against Hinton, Vinyals and Dean (2015), section 2; and the effect of always choosing the most likely text against Holtzman and others (2020).
  • Simplified on purpose. Both panels use invented probabilities for a handful of words, and treat each word as one token; the first panel looks only at the last word. A real model scores tens of thousands of possible tokens, or more, at every step, using the whole text. The split of unbelievably is an illustration, not the output of a real tokeniser. Services use temperature values in their own ways.
  • Fictional examples. The Brindle Vale Academy example (the school, Ms Okafor and the head teacher) is invented.

Further reading