How LLMs work — what a designer actually needs to know

Most explanations of how large language models work are written either for engineers who want to build one or for civilians who want to be amazed by one. Neither of those is me. I'm a product designer. I needed something in between: enough mental model to design AI features that don't break trust, and enough vocabulary to prompt the things effectively without treating them like magic.

This is the version I wish I'd read first.

What it actually is

A large language model is a system trained on massive amounts of text. From all that reading, it learned to recognize and reproduce patterns in language. That's the whole story. It isn't a database, a search engine, or a reasoning system — it's a statistical pattern-matcher that predicts the most probable next word given everything that came before.

If you internalize one sentence, make it that one. Almost every weird thing an LLM does — the confidence, the hallucination, the way it sometimes "agrees" with whatever you said last — comes from this single fact.

How a single response gets generated

Every time you send a prompt, the model looks at all the text in its context window — your message, the conversation history, any documents you pasted in — and generates a response one token at a time, based on learned patterns.

It has no memory outside that window. No permanent storage. No awareness of your previous sessions, your previous projects, or what you told it yesterday. The window is the entire world the model can see in that moment.

Two things directly shape output quality. The first is your prompt: the more specific context you give, the narrower the probability space, and the less the model falls back on generic patterns. The second is temperature — a setting that controls how random the output is. Low temperature produces predictable, conservative responses. High temperature produces varied, creative ones. Neither is better. It depends on the job.

The thing I had to keep reminding myself: the model doesn't retrieve facts. It generates text that fits the pattern of how questions like yours typically get answered. That's why it can be confidently wrong. There is no internal flag that distinguishes "I know this" from "this sounds right." From the inside, those two feel identical.

The terms worth actually remembering

A short list. Skim and move on.

Token — the basic unit the model reads and generates, roughly a word or a word fragment. Helpful when the model cuts off mid-thought or behaves oddly near a length limit.

Context window — the model's working memory for this session. Everything it can see right now. No context window, no situation awareness.

Temperature — output randomness. A UX decision, not just an engineering one.

Hallucination — confident, plausible output that isn't grounded in fact. Happens because the model generates rather than retrieves.

RAG (retrieval-augmented generation) — the standard way to ground a model in real data: inject the relevant documents into the context window at query time. The alternative is retraining, which is expensive and rare. Almost every "AI assistant on top of company X data" you've ever seen is doing RAG behind the scenes.

System prompt — the hidden instructions a product team gives the model before the user ever types anything. You're designing this when you spec an AI feature. It shapes everything.

Sycophancy — the model's tendency to agree with you unless explicitly told not to. Dangerous for design critique. The remedy is an explicit anti-sycophancy instruction.

What I do with this as a designer

Four things changed about my actual work once this clicked.

I prompt better immediately. Vague input activates generic patterns. The fix is always upstream — more context, tighter constraints, an explicit format. Not rephrasing the same weak prompt three times.

I can spot hallucination risk in AI features early. When a team wants to build an AI assistant on top of company knowledge — HR policies, support documentation, product manuals — I can name the core risk: without proper grounding (the RAG layer), the model generates plausible-sounding answers from training patterns, not from the actual source. That's a trust-and-safety problem worth raising in design review, not a "tech issue" to leave to engineers.

I treat the context window as a design constraint. In any AI-powered feature, the model only knows what's in the window at generation time. If the right information isn't there, the feature fails or fabricates. Designing what gets injected into that window — and when — is a product design decision. It's no different from designing what data a form field can access.

I design AI errors as expected states, not edge cases. Because the model can't reliably flag its own uncertainty, the UI has to do that work. Confidence indicators, fallback states, source citations, freshness signals — that layer is what makes an AI feature trustworthy. In B2B and especially in banking, that layer is what separates an AI feature people use from an AI feature compliance shuts off in week two.

The honest postscript

I'm not an ML engineer and I have no interest in becoming one. The point of this model wasn't to make me technical. It was to make me dangerous enough to ship AI features that respect the user — and to push back, in the room, when a feature is being designed in a way that assumes the model can do something it can't.

That turned out to be enough.