Hallucination & tool use

Running the model & the world

The idea

A model never consults a store of facts. It samples plausible continuations - in the same confident register whether the fact is baked into its weights or invented on the spot. Hallucination isn’t a bug in retrieval; there is no retrieval.

The mitigations that work are behavioural. First, probe the model to find where its real knowledge ends, and train an honest “I don’t know” exactly at that boundary (Karpathy walks the interrogation recipe at ~1:20-1:33). Second, give it tools: a search or code call pauses generation, runs outside the model, and splices the result into the context - where knowledge is exact and directly usable, unlike the lossy recall of weights (~1:33-1:41).

Verdict

What it fixedProbe what the model actually knows, train an honest refusal at that boundary, and let tool calls splice fresh, checkable tokens into the context.

What it still costsA tool answer is only as good as the context that holds it - and the model can still talk over it.

The story, animated

Same mechanism, two outputs - ask it twice to catch the fabrication, then watch a tool splice in something exact.

Where it sits

Worth having first: Context moves vectors · Sampling · Instruction tuning & RLHF

Unlocks: What it can’t remember

Source-checked against Karpathy, “Deep Dive into LLMs like ChatGPT” (2025), chapters 1:20-1:46.