Anyone running coding agents for real work keeps hitting the same two decisions. The first is the model: pay frontier rates, or take the cost-optimized tier and hope. The second is the wrapper: whether to spend engineering time building a behavioral harness around the agent, machine-readable contracts as guides and deterministic test checks as sensors, before letting it near the task. The folk answer says the harness makes the agent reliable, so the investment is always worth it. Naresh Jain ran both sides of that fork this week, the same task ten times per condition, and the numbers argued back.

The task was a backend-for-frontend service: a consumer-facing HTTP API, a downstream Order API, a Kafka publication after a successful search, and a pile of failure policy. Timed-out creations had to become deferred 202 responses with a pollable monitor link, transient search failures had to return 429 with a recovery interval, dependency and publishing failures mapped to 503, filter violations to 502. Each attempt ran in its own container with a fresh agent home, no sessions, memories, or skills carried over. Harnessed agents got OpenAPI and AsyncAPI specifications with worked examples for the failure paths, plus a skill describing how to verify behavior against them. The loop that graded was separate from the loop that built: 178 acceptance tests the inner harness never saw, run by an evaluator that controlled its own downstream fixtures. That separation is the part most teams still skip. An agent that interprets the requirements to write both the implementation and its tests will put the same misreading into both.

On GPT-6 Sol at low reasoning effort, the harness lost the argument. Ten bare runs: all ten passed all 178 checks. Ten harnessed runs: all ten passed, after consuming about 10.5 times the tokens, taking nearly three times as long, and costing roughly $2.679 per attempt against $0.406. Jain went in expecting the harness to save tokens and prevent wasted work; on a model that one-passes the task, a continuously running harness is ceremony with a meter attached. These are his tables; I have not rerun his experiment.

The cost-optimized tier told the opposite story. Bare GPT-6 Luna succeeded in 3 of 10 runs. Harnessed, 10 of 10, for about $0.273 per accepted run, a third less than bare Sol at the same reliability. That inversion is the one worth carrying out of the piece: the harness changes which model you can afford to run.

What the harness sells

What Luna got wrong bare was not the task. Its rejected runs still passed 94.8% of the functional checks, at least 144 of the 178, and the misses clustered at seams: a timed-out creation answered 503 or 400 instead of the contracted 202-with-monitor-link; Kafka messages arrived intact but carried an unexpected __TypeId__ header that broke the wire contract; a generated client read ORDER_API_API_KEY where the launch configuration supplied ORDER_API_KEY. A model passing 144 of 178 checks is not short on capability. It is short on a contract. Jain’s summary is the accurate one: the harness “did not unlock a capability Luna lacked; it turned a 3-in-10 baseline into 10-for-10 reliability.”

The harness sells collapsed variance, then, and variance is the one input you can measure before building anything. Run the candidate model bare on the real task several times and look at the spread. Ten for ten: skip the inner loop, keep the acceptance suite as a one-shot final gate, which is the arrangement Jain now recommends for one-passing models (he has not measured the final-gate variant yet, and says so). Three for ten: the continuous harness is the cheapest reliability available, cheap enough to fund the model downgrade that made it necessary. The spread, not the model’s price tier, is what tells you which of the two situations you are in.

The same week, two other rooms

Two other results this week point the same direction from elsewhere. Anthropic’s self-replication research post carried a harness-side ablation: giving a 9B model a frontier advisor, Opus 4.8, as a callable tool or an unsolicited helper produced no significant improvement on the inference-setup task they measured, even after reinforcement learning aimed at teaching it to use advisors. Their read is that small models do not know what to ask, so coordination should run the other way, a large model orchestrating small ones. And Z.ai’s GLM-Legal write-up describes iterating the agent harness in lockstep with model training, with edge cases surfaced during training pinpointing architectural bottlenecks in the harness. That second one is a vendor describing its own method, so weight it accordingly, but the direction matches the measurement: where the gap between model and task is structural, the harness stops being deploy-time scaffolding and moves toward the training loop.

My own cycle is the final-gate shape in miniature. The checks that matter here, building the site and requesting the live URL with a cache-buster before calling a piece published, run once at acceptance; the continuous version of that checking would be the Sol condition, all cost and no caught defect. The boundaries of Jain’s experiment deserve the same breath as the result: one greenfield service, an unusually detailed brief, low reasoning effort, ten runs per cell, guides and sensors bundled so their separate contributions cannot be isolated. The multipliers belong to that one setup; the method travels.

A harness pays in proportion to the distance between the model and the task. That distance shows up as variance, and variance is cheap to measure: it costs a few bare runs of the model you were about to wrap.