How do language models work?
One token at a time.

Predict what comes next, then do it again.

Text is split into tokens—often words, pieces of words, punctuation, or other symbols. Given a sequence of tokens, an autoregressive language model estimates which token is likely to come next. The system selects a token, adds it to the sequence, and runs the process again. For the details behind that split, read the guide to tokens and context windows ↗.

For a familiar sentence like “The city woke before…”, many continuations could fit. The model produces scores for possible next tokens based on its learned parameters and the context in front of it. The choice is repeated, so a small difference early in the sequence can lead to a different answer later.

Many widely used language models use Transformer architectures. The 2017 Transformer paper introduced an architecture built around attention mechanisms, which help the model relate different parts of a sequence. The word “Transformer” describes a family of architectures; products can use different training and serving setups. See how the architecture works ↗.

First the model learns patterns. Later, it learns how to respond.

GPT-3 was presented as an autoregressive language model that could respond to task instructions and examples placed in text, without changing its weights for each task. This is called in-context learning. A prompt may change the current input; fine-tuning changes model parameters through a separate training process.

Instruction-tuned systems add another layer. InstructGPT, for example, trained models using demonstrations and human preferences. That can make outputs more useful for following requests, while leaving separate questions about truth, uncertainty, and performance outside the tested cases.

For a step-by-step view of prompt processing, token generation, and model serving, continue with what happens during AI inference ↗.

The visible sentence may be only part of the context.

A product can send the model system instructions, earlier messages, retrieved passages, structured data, or tool results alongside your request. The model predicts from the context it receives; it does not automatically know which parts are current, authoritative, private, or trustworthy.

A context window is finite. Older material can be omitted or compressed when a conversation grows, depending on how the product is built. Persistent memory is another product feature: it requires the application to store and supply information again. A model’s learned parameters are not, by themselves, a live database of facts or a traceable source index.

Likewise, a model does not inherently browse a website, call an API, or send an email. An application can connect it to those tools and decide which results to return. The boundaries and permissions belong to that surrounding system.

Explore the connected parts: memory ↗, retrieval and fine-tuning ↗, and agents and tools ↗.

Generation has choices and uncertainty.

The system may select the highest-scoring next token or sample among likely alternatives. Settings such as temperature can change how much variation the sampling step allows. The exact response can also change with model version, system instructions, available context, or tools.

That variation can make writing flexible, but a likely continuation is not the same as a verified fact. The model can produce a plausible answer when it has missing, outdated, or conflicting information. A polished explanation does not show where each claim came from.

When accuracy matters, check the evidence directly. See why fluency is not evidence ↗ and how benchmark results should be read ↗.

Define the job before polishing the prompt.

1. Describe the task and audience. Make clear what the response should help someone do.
2. Supply relevant context. Include the source material the model should use, and label content it must treat as untrusted.
3. Set constraints. Ask for an appropriate format and what the system should do when information is missing.
4. Check important claims. Compare them with current primary sources or a trusted record.
5. Keep consequential actions reviewable. Decide which steps need a person’s approval and what tools the system may access.

For a practical prompt structure, read a prompt is a brief, not a spell ↗. To compare model options on your real task, start with the work and its evaluation criteria ↗.

Follow the whole system.

Start with the AI field guide ↗ for a map of models, tokens, memory, retrieval, tools, and evaluation. Then follow the topic links to see how those parts work together.

To understand how a broadly trained model can support many kinds of work, continue with the guide to foundation models ↗.