Train, monitor, manage · Lesson 6 of 6
What do you do when an Agent gets it wrong?
Published September 8, 2026 · Updated September 25, 2026
Open the run from the case file, read what Proctor discarded, and correct it where the work is: Send back on the plan, Request changes on the pull request, or a reply in Slack. Corrections take effect on the next step. Run again after a failure, and set a line back to supervised when the record says so.
Why does this lesson exist?
Because Agents get things wrong, and a product that pretends otherwise is a demo. What matters is what a miss costs you: thirty seconds and a note, or an afternoon of log archaeology and a support thread.
Everything below is designed for the first one. An Agent you can correct in the moment, with the correction sticking, is a teammate.
How do you work a miss, step by step?
- Open the run from the case file. Every step on the case file links to the run behind it: what the Agent read, what it changed, which Playbook rules applied, which decisions it grounded itself on, and what it cost. You start from evidence, not a guess.
- Read what Proctor discarded. Proctor considers far more than it posts. The findings it dismissed sit one click behind the review, each with its reason. A quiet review you can check is worth something; a quiet review you cannot is not.
- Correct it where the work is. On a plan, Send back with a note for the next pass. On a pull request, Request changes with your notes, which starts a fix round on the ticket’s own line. Or reply in the Slack thread or on the pull request itself. The correction lands on the ticket, and the next step reads it.
- Let the correction become a rule. If it should outlive this ticket, the Team proposes it as a Playbook rule. A proposed rule is already active; adopt it behind the Playbook card on the Team page to settle it, or dismiss it if you meant it only this once.
- Run again after a failure. A provider outage or a flaky sandbox is Run again on the case file, not a support ticket. The run resumes with the same context, and the failure stays on the record. A run a deploy interrupted resumes on its own.
- Set the line back, or ask @yagni. If the record no longer supports the trust you gave a line, open it from the Team page and set its Authority back to supervised. And when you want a step explained rather than fixed, ask @yagni in the rail beside the case file: it knows the ticket, the Team, and the decisions behind it.
What makes a correction stick?
Where it lands. A correction typed into a chat window is a message. A correction on the case file or the pull request is part of the ticket’s record, so the next run reads it as context and the review reads it as a constraint.
That is also why corrections are worth making carefully. They are the raw material of both the Playbook and the track record a person reads before moving a line.
When should you change a line’s rung?
When the pattern, not the incident, has changed. One bad plan from a line with fifty good ones is a note. Three in two weeks is a reason to set it back to supervised.
A reversal of something a line shipped on its own is one more count on its record, beside accepted as proposed, edited, and sent back. It changes no level by itself; that call stays yours, with the record beside the blank. Merge was a person’s click anyway, at every rung.
What if it keeps getting the same thing wrong?
Then the rule is missing, not the intelligence. Look at what the run read before it worked: if the answer was never written down anywhere, the Agent was guessing in good faith. Write the rule in plain words, scope it to the repository or path where it applies, and the next plan starts from it. If it is a point of view rather than a convention, put it in the line’s instructions instead. That is the loop the whole product runs on, and the Daily Brief will tell you the morning it holds.
Common questions
How long does a correction take to stick?
It applies at the next step. A plan sent back is re-planned with your note; a pull request with changes requested gets a fix round with your notes. Nothing waits for a retraining cycle or a nightly job. If the correction should apply beyond this ticket, the Team proposes it as a Playbook rule and that rule is active while it waits for you to adopt or dismiss it.
Can I see what a reviewer chose not to say?
Yes. Proctor posts one review, but the findings it weighed and dismissed are one click behind it, each with the reason. That is what makes a clean review meaningful: you can check that it looked, not just that it said nothing.
Does a bad run cost me the whole ticket?
No. The ticket is the record, and a bad run is one step in it. Correct the step and the work continues from there, with the miss left on the record rather than erased. The stage pie shows a failed verdict as a red wedge until the next round clears it.
What if the Agent was right and I was wrong?
Dismiss the rule you were about to write and let the run stand. Nothing forces a correction, and a line is not penalized for work you accepted. A reversal of something it shipped on its own is one count on its record, and only a person changes its rung.