Why Coding Agents Write Narrow Fixes (And Why Logs Won't Save Them)

Before I introduce variant verification, let me tell you who I am and why we built this. I have worked in financial industry for about half-a-decade, and when I recently build HFT infrastructure from scratch, we never merged a bug fix without replaying recorded traffic variations against it first. While building that, I found certain patterns that are helpful for me, and also helpful for agents, to debug faster, to reproduce the issue exactly, and to verify the fixes.

Variant verification¹² is about replaying the event variations which govern transitions in your state machine. What is a narrow fix, you may ask? A narrow fix is a software's incomplete patch which determines whether only the recorded crash passes while sibling events still fail.

Why does a variant check help to catch narrow fixes faster? Whenever we are stuck reviewing a bug fix pull request by looking at unit tests that the agent wrote itself, a simple variant check lays out in bare bones the events that took place, which caused the software to fail on sibling inputs. Let's say you have a bank account. Your bank account is a summation of all the credits and debits of your money. Now imagine one day you go up to your bank, and it says that you have 10 dollars. When you checked last night, you had 100k. So you will demand the transaction ledger for this, right? You are not ready to accept a green unit test when a deposit patch still leaves transfers broken.

{"id": "evt-008", "type": "deposit", "account": "bob", "amount": null}

if event.Type == "deposit" && event.Amount == nil {
    env.Emit("ledger.rejections", Rejection{Event: event.ID, Reason: "missing amount"})
    return nil
}

A variant verifier is kind of like that. It has a transactional ledger which keeps account of all the state changes, all the event variations that lead up to the same crash. In a way, it makes the generality of a bug fix auditable.

Now, since I have worked in the finance industry, I see its applications everywhere. It helps to catch narrow patches, to actually see what variations break the fix, and so on.

Why deterministic variants matter#

Starting from a snapshot, if you keep replaying the variations that are in the journal, your fix should hold on every step. That's the ultimate goal for Kavach³⁴. What makes it challenging are certain shortcuts sprinkled through agent patches. Shortcuts come from hardcoding an event ID. They can come from guarding only one branch. They can come from the fact that the agent is rejecting valid inputs too.

Having the ability to replay 56 variations deterministically in milliseconds is one of the key guarantees of our verifier, and that is what makes the agent loop so reliable.

Challenges of verifying agent bug fixes#

These challenges all come down to how agents optimize for a passing test. Verifying an agent's patch has several such traps.

Guarding only one branch. Can a single replay catch a narrow patch? Catch every missing guard? How does it know what happens on a transfer? Which other branches share the same bug? If these variations aren't tested, the same bug will crash production the next day on a different event type. As a company, Kavach Labs is going to tackle these challenges so that narrow fixes get rejected automatically.

Variations that don't reproduce the bug. Software often behaves differently when you change a field value. Say you design a check that replaces a missing amount with 5,000 dollars. How do you score that test without running it on the old build first? With 5,000 dollars the old build works fine, so on replay you are testing a happy path and not the crash.

Patches that break valid traffic. One system requirement for a good fix is that earlier valid events should produce the exact same outputs as before. Most careful human patches follow this principle, but quite a few chunks of agent code still don't. How do you catch those? These are the uncomfortable parts of the system that we want to tackle.

claude mcp add kavach -- kavach mcp
npx skills add kavachlabs/kavach

We're building Kavach Labs to tackle this problem head-on. We love deterministic verification as an architectural concept, and we want agentic software to be written and deployed so that it's easy to capture a bug, replay it, fix the code, and deploy the fix with more confidence.