Session 7 — Failure analysis
Goals
- Separate a tool failure from a wrong answer.
- Decide when another attempt helps.
- Spot a convincing answer that hides an action that did not succeed.
- Tell a check that brings new evidence from asking the model to "check itself".
What to check in the log
For every step: what the goal was, which tool was called, what came back, and whether the counter moved. "The agent behaved strangely" is not a diagnosis.
A tool failure is an exception, a timeout, or output that does not match the schema. A wrong answer can come even when all the tools succeeded: for example, a correct calculation on a file from the wrong year.
A silent failure is the hardest to find: the tool returned an empty result, and the agent went on as if it had an answer. So an empty result gets its own status, not an empty list that looks like "no problem".
Another attempt, and when not
Another attempt fits a transient fault, once, with the error message inside the context. It does not fit a permission refusal. An action that sends an email or writes a price needs an identity (idempotency) key, otherwise the second attempt may duplicate it.
An idempotency key is a fixed id for a request. If the same request arrives twice, the tool recognizes it and returns the first result instead of acting again.
A loop that repeats the same call with the same arguments is a sign to stop. A budget keeps it from running forever, but only an explicit rule catches it early.
A check with evidence
"Check yourself" without new evidence asks the same calculation to repeat itself. A check in code, or an external source, is evidence. A rephrasing is not evidence.
When the checker is another model, it may fail in the same place as the first, because both read the same context. A useful checker sees something the first did not: the result of a check in code, another source, or a rule written in advance.
Read the following log and write one line: which step failed, whether the failure is in the tool or in the answer, and what the right status is.
1 retrieve {"q": "recall"} ok ids=["A:1"]
2 retrieve {"q": "recall"} ok ids=["A:1"]
3 calculate {"recall": 0.95} ok value=0.95
4 finish "meets the requirement"Check your understanding
Reading
Shinn et al., Reflexion: Language Agents with Verbal Reinforcement Learning, NeurIPS 2023. arXiv:2303.11366. The abstract and figure 1. Notice where the signal that tells the agent it was wrong comes from.
Summary
Found a mistake on this page? Email the lecturer