Session 8 — Knowledge, memory and RAG

Session 8 of 13TheoryDraft5 min read

Draft for the lecturer's approval.

The explanation of QuickScreen is based on the lecture deck prepared for the topic. Its place in session 8 follows the site's sequence, and it must be confirmed against the syllabus in Moodle.

Goals

• Distinguish context, working state and stored memory.
• Explain why a retrieved passage is not an answer, and why an empty retrieval is not proof that the information is missing.
• Decide when to retrieve and when sending the whole text is enough.
• Spot a stored note that has gone stale.

Three roles of information

Role	What it holds	Example
Context	What is sent with the current request	The question, P:1 and C:1
Working state	What the application tracks during the task	Ids already checked
Stored memory	A note for next time	"We used P:1 and C:1"

A stored note does not enter the model by itself. The application loads it, checks the version, and fetches the passages themselves. A note that says "recall is 95%" stops being valid after a correction that says 36 out of 40.

Retrieval is not an answer

Sources are split into passages with a stable id. Keyword search finds an exact term. Vector search finds similar wording, and can also bring an opposite claim close. After ranking, you read the passage.

Retrieving one passage does not prove that the rest of the collection is empty. Only after checking the allowed set may you say it holds no evidence. And even then, you do not conclude that the missing value is zero.

An agent decides whether to retrieve, whether to retrieve again, or whether to answer. A system that always retrieves the same way does not demonstrate this decision.

Retrieval or the whole text

If all the sources fit in the context window, you can send all of them and skip retrieval. It is simple, and there is no risk that the right passage was not ranked high. The price is the cost of every request, and a model that has to find the relevant sentence itself inside a long text.

Retrieval pays off when the collection is large, when it changes, or when you want to know exactly which passage each claim came from. Either way, the remaining question is the same: does the claim in the answer appear in a source that was sent.

A deterministic simulation of a pre-written policy. There is no language model here, and no real email is sent.

Passages (k)

2

Call budget

6

Goal · step 1 of 9

Goal

Does QuickScreen meet both protocol requirements?

Status: running · passages: — · tool calls: 0 · 6 left

Choose "Missing cost" and compare it with "Two requirements". Then lower the budget to 1 and see a stop, not a refusal.

Check your understanding

The search returned only A:1, which says recall is 95%. What may the agent say?

Pick an answer to see the explanation. There is no grade here.

Reading

Lewis et al., Retrieval-Augmented Generation, NeurIPS 2020. arXiv:2005.11401. The abstract, figure 1, and the opening of section 2.

Summary

What stays. Context, working state and stored memory are three different things. A retrieved passage is evidence to read, not an answer.

What breaks. An empty retrieval that turns into “no information”. An old note that wins over a correction.

Next: session 9, Document agent

The mark is saved only in this browser. It is not an academic record and is not sent to a server.