A model can produce a clear answer that is wrong. This video explains why that happens and what we can do about it. We'll look at hallucinations, the knowledge learned during training, and the information supplied in the current context. We'll also look at the knowledge cutoff and at errors these limits can cause in code.
Underneath, the model is a completion engine. You give it some text, and it predicts how that text most plausibly continues. Here we feed it the start of a sentence about a Haruki Murakami novel from the year twenty thirty.
The model finishes the sentence with a specific title. Notice how fluent and confident it sounds. It reads exactly the way a genuine fact would.
The title in this example is made up. It is presented as the name of a published book, even though the sentence names the future year twenty thirty. The title was simply invented. It looks like a fact, but nothing real stands behind it.
The model predicts continuations that fit the text. A plausible book title can fit this sentence even when the book does not exist. Predicting a continuation does not, by itself, check whether that continuation is true. The beginning of this sentence strongly suggests that a title should follow. Saying I don't know is possible, but it does not follow the pattern the sentence sets up. That is why a model can produce a fluent guess when it lacks the fact we need. An invented or incorrect claim presented as fact is called a hallucination.
There is one more part to this story. The raw completion engine is not what you actually talk to. Before a model is released, it goes through a later phase of training that shapes how it behaves, and part of that work is teaching it to say, I don't know, when it knows something poorly. This can make uncertainty easier to express, but it does not eliminate hallucinations completely. A paper by Kalai and colleagues points to one reason they persist. A benchmark that rewards a lucky guess and gives no credit for admitting uncertainty encourages guessing. High scores can therefore coexist with confident errors. We'll look at that later phase of training in week three.
So where does the model's knowledge actually live? We need to distinguish two sources: what the model learned during training and what the current request supplies. At the center is the model itself.
The first is knowledge stored in the model's weights. We call it memory here, but it is different from the instruction files the harness reads. The model is a neural network, and training on large amounts of text changes its weights. Week three has a video that opens up that network and how it is trained. The weights capture a great deal, but they are not a database of exact facts. A model can recall a common fact correctly and confuse a rare name, date, or detail.
The second source is context: the information supplied in the current request, including your prompt and anything the harness adds. The model can refer to this text while producing its answer. With a source in the context, the model can use the actual text instead of reconstructing it from its weights. It can still overlook or misunderstand a detail, especially in a long context. Give it the relevant source, and then check that its answer agrees with that source. Put differently, don't make the model recite from memory something you could simply have shown it directly.
A model's knowledge cutoff marks how recent its training information is, shown here by the amber tick on the timeline. The left side shows information that could have been included in training. For events after the cutoff, we need another source.
Even before the cutoff, what the model learned is uneven. The model may have learned some facts from this period, missed others, or learned conflicting accounts.
For something newer than the cutoff, don't rely on knowledge from training alone. A library released last month may have different functions from the version the model learned about. Supply the documentation for the version you are actually using.
The practical response is to bring the newer information into the context. You can provide a source yourself or have the agent retrieve it with a tool. That puts the source in front of the model, so it can read the new information instead of guessing from older training knowledge. The same applies to your code. It was likely never in the agent's training data, so it must be read into context for the agent to work with it.
These limits also show up in code. Suppose we ask for code to reshape a pandas DataFrame and suggest a helper for reading C-S-V files.
It answers confidently, with a method called unpivot and a package called kewl-utils. At a glance, both look perfectly reasonable.
Neither name works. Unpivot is the name in other DataFrame libraries, but pandas calls this operation melt. The package name is invented. Your normal tools catch both. The script fails on its first run, and the install fails for the package. But a clean install alone does not prove it is the package you meant.
This is the same recall problem as the invented book title, now with names from a library. Give the agent the documentation and types for the version you use, then check the code with your tools. How to build those checking loops well is a topic we'll come back to later in the course. In a study by Spracklen and colleagues, about one in five suggested packages did not exist, across the models and tasks they tested.
Let's recap. First, there's hallucination. A plausible continuation can contain a false claim. Training can encourage the model to admit uncertainty, but fluent wording is still not evidence of truth.
Second, there's the difference between memory and context. Providing the relevant text reduces reliance on uncertain recall. You still need to check how the model uses it.
Third, there's the knowledge cutoff. Training does not keep up with new events or new library releases by itself. Bring current information into the context.
In code all of this shows up as invented APIs and packages that don't exist.
Notice that every one of these fixes points in the same direction. Put what matters into the context, and verify the output afterward. The next video applies this to writing prompts: giving the model a clear task and enough information to work on it.