Calyx AI · October 21, 2026 · 6 min read

The self-annealing loop: make every AI agent failure leave a test behind

When an AI coding agent hits an error, the easiest move is to get past it: retry, reset, reprocess. That makes today's run green and guarantees the same failure next week. The self-annealing loop is a short rule that changes what the agent does after something breaks, so the system gets harder to break each time.

Why agents default to recovery

Ask a coding agent to finish a job and you have told it, implicitly, that a finished job is the goal. When a step fails, the shortest path back to "finished" is recovery: run it again, clear the stuck queue, restart the service, add a retry. Each of those is reasonable in isolation. Together they hide the cause, and because the next session starts with a fresh context, nothing about the failure survives except the workaround.

A human engineer does the same thing under pressure. The difference is that the human usually remembers. The agent does not, unless you give it somewhere to write the lesson down and a rule that says it must.

The loop, in order

Annealing is the metallurgy term for heating and cooling a metal so its internal stresses relax and it ends up tougher. In an agent-run system the "heat" is a failure. The loop is five steps, and the order is the whole point:

1. Diagnose the root cause. Keep asking "why" until you reach something that, once changed, prevents the failure from recurring. "The request timed out" is a symptom. "The health check treats any timeout as a store outage" is a cause.
2. Fix it in code, config or infrastructure, not in the data.
3. Add the test that would have caught it, so the fix cannot silently regress.
4. Write the standard down. Add a line to the directive's Edge cases & learnings section, and a lint if the pattern can be detected mechanically.
5. Then recover: reprocess the stuck items, restart, re-run.

Recovery still happens. It just happens last, after the cause can no longer produce the same failure. A rule worth adding: if the agent cannot finish steps 1 to 4 in the session, it says so and leaves the item open instead of quietly recovering.

Put it where the agent reads it first

The loop only works if it is part of the agent's standing instructions, not a prompt you remember to type. In a D.O.E. repo (Directive, Orchestration, Execution, described in the first article) it lives in CLAUDE.md. A condensed example:

## When something fails (self-annealing)
1. Diagnose the root cause. A symptom is not a cause.
2. Fix it in code/config/infra.
3. Add the test that would have caught it.
4. Add the standard to the directive's "Edge cases & learnings"
   (and a lint if it is mechanically detectable).
5. Only then recover. Never recover first and call it fixed.
Record each pass in the anneal log: date, symptom, cause, fix, test.

The anneal log matters more than it looks. It is how you see that the same area keeps failing, which is usually the signal that the first fix treated a symptom.

A real example: three anneal passes in four days

We run the store behind this site with a daily agent job on a small Linux box. Each day a script checks the storefront and the checkout endpoint, and if either returns nothing (HTTP status 000), it raises an ATTENTION note: the agent must diagnose and fix the cause before doing anything else.

Pass 1 was a real fix and still not the whole answer, which is normal. What the log gives you is the history: by pass 3 it showed the same alarm with the same kind of cause, which pointed at the check itself rather than at the store. Recovering first would have meant treating a healthy site as broken.

The last row also shows the loop's honest exit: some root causes are not code. When the cause is a flaky network link, the right standard is "classify it correctly and tell a human", not another retry.

What to check before you call a pass done

• Could you state the cause in one sentence that does not contain the words "sometimes" or "flaky"? If not, keep asking why.
• Does a test fail on the old code and pass on the new? If you cannot write one, write the lint or the directive line and say why.
• Is the learning in the file the next session reads first, not in a commit message or a chat transcript?
• Did recovery happen after the fix, and was it verified (a 200, a matching hash, a row count)?

Start with one directive

You do not need a framework to try this. Add the five-step block above to your CLAUDE.md, .cursorrules or AGENTS.md, and give your most failure-prone job a directive with an Edge cases & learnings section. After a week, read that section. It is a list of failures that now have a test, a rule or a lint against them.

The free Agent Ops Kit Lite (MIT) has the CLAUDE.md with this loop, the directive template and a secrets linter. It also installs as a Claude Code plugin: /plugin marketplace add skibuff22/agent-ops-kit-lite.

Written by Calyx AI, a product line of Quality Tester Solutions LLC. This article was drafted with AI assistance; the incident table is taken from our own run log.