LLMs Don't Hallucinate — You Just Don't Ask Them to Show Their Work

I’m involved in some legal processes for my father that died. I used to collaborate with my lawyer, and whenever I receive something, I double check it with the LLM — the next step, an update, and so on. And sometimes, yes, it hallucinates.

But keep reading.

(This is one of three pieces from the same brain dump — the other two cover permission modes and context management. This one’s just about evidence.)

LLM Is Not Hallucinating

Let me start with this: LLM is not hallucinating.

And you’re saying — are you serious? What the fuck?

That’s the reaction. But that’s the reality, once you ask the LLM to double check your data. To actually stop it, though, you need to know why it happens in the first place.

The Ugly Truth

Quick, high-level version here — not 100% technically precise, but enough to get you the ugly truth.

An LLM is trained up to a certain point, and then training stops. Even Fable — the model everyone was so hyped about — stopped training in January 2026. Technology keeps moving. The model doesn’t.

Now add your own context on top of that — your documents, your prompt, your history. That context is giant. Physically, it’s impossible for an LLM to pull all of it into memory and reason over the whole thing at once.

So what does it actually do? It picks a slice. It uses embeddings to find the section of your context that looks closest to what it needs, and it starts generating from there — token by token — until you get a sentence, a reference, an answer.

That’s the mechanism: it’s retrieving the closest match, not recalling the exact truth.

And if the embedding picks the wrong slice — because it didn’t check something online, because the data’s not updated, because of a dozen other reasons — you get a hallucination. Not a malfunction. A retrieval miss.

The Two Steps

Back to the legal documents. Here’s what I actually do — two steps.

Step one: expand the context. Before it answers, I tell it — hey, don’t just work off what you already know, go retrieve the updated stuff, search online, pull in the real source. That’s how you get today’s information back into its working context, instead of whatever it remembered from training.

Step two: validate. Once it answers, I ask it to double check everything — check resources online, make sure you only have real data, and prove it. Show me the sentence, show me where it came from, give me the URL if there is one.

I do the same thing with my own product, by the way. Sometimes people ask, hey, is this true, was that okay — can you find evidences.

The PDF Trick

This is even useful for writing better prompts. Another example: you’re reading from a PDF, extracting some information, and asking the LLM to summarize it.

Now the LLM will summarize something that’s different. And you don’t know why.

Small trick: ask the LLM to summarize, and also provide you the evidences — the facts, the pages, which PDF. It will hallucinate way less. Same idea as step two, just for documents.

Even Humans Hallucinate

So, hallucination for me is just you not giving the LLM the power to double check.

And to be candid, even humans hallucinate. Ask someone something generic and they’ll reply fast, from what they know — their best guess, not always right. Stop them and say, wait, I don’t think that’s true, prove it — now they need more time, but they’ll go find the receipts.

Same thing with an LLM. Ask it something generic, it answers fast from what it already has. Ask it to prove every sentence, it slows down — but it gets it right.

So please don’t get me wrong: LLM doesn’t hallucinate. You just didn’t ask it to prove its work.