case studies

How a Coding Agent Tries Three Fixes at Once and Ships the One That Passes

A bug report comes in: Median([1 2 3 4]) returns 3, and it should return 2.5. A coding agent can usually think of more than one fix. The usual move is to pick the one it likes best, apply it, run the tests, and start over if they fail. Each failed attempt has to be undone by hand, and the working copy spends the whole time half changed.

With Multivergence the agent drafts all three fixes at once, each in its own branch, and lets the evidence pick. This case study walks through examples/fixes in the repository. The patches in it are real Go functions and the test suite really runs on them. Every number below is from its recorded output.

Stacked bars of three patches’ probabilities in under four minutes: patch-c dies at once for changing the API, patch-a drops from 62.5% to 40% after review and dies when a test fails, patch-b ends at 100% and is promoted The patches’ probabilities at each step, from the recorded run.

Three Drafts, One Rule They All Share

The current code and the bug report go on the trunk. Then the agent forks a branch per patch, and the weight it gives each one is how confident it is: 0.5 for patch-a, 0.3 for patch-b, 0.2 for patch-c. Every branch also gets two constraints. The public signature has to stay the same, and the test result can’t be “fail”.

s.Fork("patch-a", mv.Trunk, 0.5)
s.Constrain("patch-a",
    mv.Equals("api/Median", "func Median(xs []float64) float64"),
    mv.NotEquals("tests/result", "fail"))

Patch-c also returns an error for empty input, so it changes the signature to (float64, error). The moment the agent writes that signature into patch-c’s branch, the branch dies. No test ran on it. The other two now read as 62.5% and 37.5%, and the trunk still holds the original code.

A Soft Clue, Then a Hard One

The reported case passes on both live patches, so it doesn’t tell them apart. Then the agent reads the two diffs. Patch-a sorts the caller’s slice in place, which a caller may not expect. That’s a reason for doubt, short of proof, so the agent gives the review a likelihood of 0.4 for patch-a and 1 for patch-b. The engine multiplies and renormalises: patch-a falls to 40%, patch-b rises to 60%.

The full suite settles it. Patch-a passes 4 of 5 tests and fails TestKeepsInput. When the agent records tests/result = fail in its branch, the constraint breaks and patch-a dies. Patch-b passes all 5 and is the only branch left.

One Step to Land the Fix

s.Promote("patch-b")

Patch-b’s code becomes the trunk in one step, together with its test result. In a store kept in a file that step is one record, so a crash leaves the old code or the new code and nothing in between. The losing drafts are gone, and the store’s history keeps how each one ended.

The agent decided what the review was worth and the tests decided the rest. The engine’s job was to keep three versions of the work side by side, each costing only the keys it changed, and to kill a version the moment it broke a rule. Run it yourself with go run ./examples/fixes.