How LLMs Actually Work (Without the Jargon)

What an LLM actually is

A large language model is a neural network: a system built from billions of adjustable internal values, called parameters, that get tuned during training.

It helps to be precise about what "trained" means here, because it's easy to picture something simpler than what's really happening. The model isn't built from a lookup table of which words tend to follow which other words. It's shown enormous amounts of text and repeatedly given one task: predict the next piece of that text.

Each time its prediction is wrong, its parameters get nudged slightly to make a better prediction next time. Repeated across trillions of examples, that process gradually shapes the network into something with an internal, mathematical representation of how language, ideas, and facts tend to relate to one another. No single fact gets memorized as a fixed string. The pattern behind it gets reinforced enough times to become part of the model's parameters.

A useful, if imperfect, picture: imagine an extremely well-read intern who has read most of the public internet, but who finished reading on a specific day and hasn't opened a new book since. They can speak knowledgeably about almost anything they read, but they don't "look things up" by default. They answer from memory, shaped by what they absorbed, not by reciting passages word for word, but by having internalized the patterns well enough to produce a fresh answer each time.

Most of today's production models go through a second stage after this initial training too: they're further adjusted using human feedback and curated examples so they behave as helpful, instruction-following assistants rather than raw text-continuation engines. That refinement changes the model's style and behavior, but the underlying prediction mechanism described below stays the same.

Watching prediction happen, one word at a time

The phrase "prediction engine" can feel abstract until you watch it work on a real sentence. Suppose the model has generated "The capital of France is" and now has to produce what comes next.

(For readability, this walkthrough talks in terms of whole words. The section below on tokens explains the more precise unit the model is really working with.)

Internally, it isn't looking up a fact in a database the way a search engine would query an index. It's calculating a probability distribution across every plausible next word, based on the patterns encoded in its parameters: "Paris" might come out with the highest likelihood by a wide margin, with "located," "a," and thousands of other candidates trailing behind it, most landing near zero.

It then samples from that distribution, almost always landing on the strong favorite, and the word "Paris" appears.

Now extend that same process across an entire paragraph. After "Paris," the model looks at the new, slightly longer sequence, "The capital of France is Paris," and predicts what comes next after that, then the next, then the next, each prediction informed by everything generated so far.

There's no single moment where the model "decides" what the whole answer will be in advance. The answer emerges one piece at a time, each step narrowing the space of what comes next, the way a sentence narrows its own possible endings as more of it gets written.

This explains something that initially seems strange: the same question, asked twice, can produce two answers that are worded slightly differently even when both are correct. The model isn't retrieving a fixed, memorized string. It's generating a fresh prediction sequence each time, which usually converges on similar content but rarely produces identical phrasing.

Training vs. inference: built-in vs. live

It helps to separate two distinct phases. Training is the (extremely expensive, infrequent) process of exposing the model to text and adjusting it until it gets good at predicting language. This is where the probability patterns from the example above get "baked in."

Inference is what happens every single time you send the model a message: the live act of generating a response, one piece at a time, using the patterns training already established.

Training	Inference
The model learns from a large corpus of text.	The model generates a response to a user's prompt.
Adjusts billions of parameters to learn language patterns.	Uses those learned parameters to predict the next token.
Extremely expensive and happens infrequently.	Happens every time someone interacts with the model.
Creates the model's built-in (parametric) knowledge.	Uses built-in knowledge and, if available, retrieved information.
Has a fixed knowledge cutoff until the model is retrained.	Can answer about newer events only if retrieval or search is available.

Because training is so expensive, it does not happen continuously. Every model has a knowledge cutoff: a date after which it has no built-in knowledge of world events, simply because that text didn't exist yet when training happened. This is precisely why retrieval, covered in Module 6, matters. It's the mechanism that lets a model bring in information about anything that happened after its cutoff.

As an example: at the time this module was written, Anthropic's Claude models had a reliable knowledge cutoff in January 2026. Ask a model like that about your brand's product launch from March 2026 without web access, and it has no parametric memory of it, no matter how confidently it answers.

Every lab publishes its own cutoff date for each model release, and it's worth checking the current one for whichever model you're testing against, since these dates move forward with every new release.

When prediction misfires: a brief note on hallucination

The next-token mechanism explained above has a direct, important consequence for anyone doing GEO: it explains why models sometimes confidently state things that are false, a behavior commonly called hallucination.

Because the model is predicting likely continuations rather than retrieving verified facts, it will, for topics where its training data was thin, contradictory, or simply absent, still produce a confident, grammatically smooth answer. That answer is built from whatever pattern was strongest, even when the pattern doesn't correspond to anything true.

This matters for GEO in a specific way. If a model has only ever encountered a handful of inconsistent, vague, or speculative mentions of your brand across its training data, it doesn't necessarily respond with "I don't know." It may instead generate a description of your brand that sounds entirely believable and is partly or entirely wrong: the wrong founding year, the wrong category, a feature you don't actually offer.

This isn't the model being careless. It's the predictable result of feeding a prediction engine thin or noisy source material. Module 14, on building citation authority, exists largely to solve this exact problem: giving the model enough consistent, corroborated material that the strongest pattern it learns about your brand happens to be the accurate one.

Tokens and context windows

Models do not read or write in whole words. They break text into smaller chunks called tokens: often pieces of words, sometimes whole words, sometimes punctuation. This is why the earlier "capital of France" walkthrough is a simplification. The model was really predicting one token at a time, and "Paris" itself is typically made of more than one token, with each new token chosen based on everything that came before it in the conversation.

The context window is the maximum amount of text, measured in tokens, that the model can "see" and consider at once: your conversation history, any documents you've shared, and the response it is generating. Anything outside that window is effectively invisible to the model in that moment, no matter how relevant it might be.

Here's where the major production models stood as of mid-2026:

Model	Provider	Context window	For scale
GPT-3.5 (2022)	OpenAI	4,096 tokens	~3,000 words, about 6 pages
Claude Sonnet 5	Anthropic	1,000,000 tokens	A short novel and then some
Gemini 3.1 Pro	Google	1,000,000 tokens	Roughly 1,500 pages of text
GPT-5.6 Sol	OpenAI	1,050,000 tokens	A mid-size codebase in one pass

Two kinds of memory, one critical distinction

This is the single most important idea in this module. A model can answer a question in two fundamentally different ways: it can rely on parametric memory, using knowledge learned during training, or it can use retrieval, bringing in fresh information from external sources before generating a response.

Parametric Memory	Retrieval
Knowledge learned during model training.	Information retrieved when a user asks a question.
Stored in the model's internal parameters.	Comes from the web, databases, documents, or connected data sources.
Limited by the model's knowledge cutoff.	Can include newly published or real-time information.
Updated only when the model is retrained.	Updated whenever the underlying sources change.
Provides general knowledge, language understanding, and reasoning.	Provides current facts, recent events, and organization-specific information.
Useful for answering questions from what the model already "knows."	Useful for answering questions that require fresh or highly specific information.

Parametric memory is like answering from what you've already learned. Retrieval is like looking up the latest information before answering. Modern AI systems often combine both approaches, using built-in knowledge alongside retrieved information to produce more accurate and up-to-date responses.

These two pathways have very different update speeds. Parametric knowledge updates only when a new model is trained, typically every several months. Retrieved knowledge can be minutes or hours old, depending on the source.

For a brand trying to be visible in AI answers, this distinction determines almost everything about your strategy: are you trying to become part of the model's built-in knowledge, appear in its live retrieval sources, or both? Module 6 walks through exactly how retrieval works and why the answer is usually "both, for different reasons."