Start at Low Effort

Most people evaluate coding models at the wrong setting. They leave reasoning effort on the default — which is usually high — and then compare products, write off a model, or crank the slider further when something misses. That's a spend knob dressed up as a quality knob. If you want to know whether a model is any good for your work, start at the lowest effort that still thinks, and only climb when the first answer is actually wrong.

I spent a long time not noticing this, and I use these tools all day. The reason is simple: the conditions for noticing it almost never happen together. You change the prompt, the files, sometimes the model. If the patch works, you move on. Effort never gets a fair A/B, so there's no reason to look at the token count.

What the Slider Actually Does

Effort is more search on the same trained instincts. The model still looks for the bugs it already knows how to see. If it doesn't treat lock order as a live issue, extra thinking just spends longer not treating it as a live issue. If another model is tuned to notice that a test is green for the wrong reason, it often does that at low.

That extra search is not free, and it is not only "think longer then throw it away." On current reasoning models, the thoughts and the extra tool calls stay in the window. They get sent back on the next turn — mostly as cache reads, which still hit the meter. Medium calls more tools. Each grep or file dump lands in the transcript and sits there until the session compact. Longer answers compound. Turn three is already carrying turn one and two's extra verbiage.

I ran the same chain of prompts twice in the same harness, same model, fresh sessions. Low used about 71k of context. Medium used about 145k. That's not a UI glitch. An independent harness study on the same family of model saw prompt tokens roughly double from low to medium, then add much less from high to xhigh. The expensive jump is the first one, and most people never take it off the default.

Quality didn't move with that doubling. I couldn't tell the answers apart. That's the tell: the slider was spending tokens on volume in the window, not a different kind of noticing.

Why You Can't Feel It

Dashboards hide it. A monthly pool or a weekly percentage makes 71k vs 145k look like a sliver. Speed is similar, because cache reads are cheap in wall-clock and expensive on the meter. In some editors the effort control barely does anything to scores or to how the task feels — a dead knob doesn't teach you anything.

And you already have a louder signal. Switching models actually changes the diff. GPT-5.6 Sol notices things Claude Fable doesn't, and the reverse, regardless of effort. That's expected. Different models are different priors: what they treat as a bug vs a smell, whether they patch the call site or walk up a layer, how aggressive they are about editing the file next to the one you asked about. It's closer to asking two senior engineers than to asking one engineer to think harder.

Harder thinking helps when the model already has the right hypothesis and needs to work it through — a subtle AST rewrite, a numeric invariant, a race that only shows up if you simulate the ordering. It does almost nothing when the hypothesis isn't in that model's repertoire. If you mostly do glue, CRUD, and "make this compile," you will never feel the knob. That's most professional coding. The model at low is already saturated.

When Low Is Enough, and When It Isn't

Low is enough when there is a local pattern to copy. Add a field and wire it through. Clone a test. Ask where something is defined. Rename, extract, flip a flag. One-file bug with a stack trace that names the line. If the answer is in the file you already have open, low and xhigh produce the same diff. High just thinks longer about a change it was going to make anyway.

Climb when the first patch is wrong in a way you'd only catch by comparing, not by vibes:

The invariant the tests don't cover. A naive canonicalize greps the strings and breaks a later pass that still needed the originals. Higher effort is more likely to find the one function that is supposed to own the rewrite — if the model already knows to look for an owner.

The error is a red herring. The compiler says type X is not assignable to Y. Low patches the call site. The real bug is a generic bound two layers up. You'll ship the hack and hit the same wall on the next feature.

Concurrency. "This test is flaky." Low slaps a wait on the assertion. The actual bug is two effects committing in the wrong order. You cannot see that from one green retry.

Authz sitting in the wrong layer. Low copies the handler and forgets the org check that lives in a different middleware. Tests pass because the test client is already scoped.

Escalating is not monotonic. Some failures get worse at high: the model "fixes" the request and regresses a neighboring test by editing more broadly. If the new miss is more editing, not a better diagnosis, stop climbing.

The Ladder

The way to test a model for your own needs is boring, and that's the point. Isolate one variable at a time.

New session. Same prompt. Lowest effort first.

If it does what you need, you're done. Don't spend more thinking on a patch that already survived the check you actually care about — tests, the diff, the invariant, not "it looks fine."

If it messes something up, same prompt, next effort, new session. Bumping effort in the thread that already failed measures "high effort plus a pile of wrong context," not high effort. The failed transcript is the contamination.

If it still can't do it at the highest reasoning level, it's probably not the model for that job. Switch models and start at low again, same prompt, new session. Don't start the new model at high just because the last one needed it. The cheap second opinion is another model at low, not xhigh on the first one.

And don't climb the ladder for "it didn't open the right file." That's a context miss. Point at the file and stay on low. Effort will not invent a retrieval habit the model doesn't have.

A Test You Can Run This Week

Next time something fails on the first try — not "I tweaked the prompt," but "the patch was wrong" — rerun the exact prompt at low and at high on fresh sessions. Diff the diffs. Then run the same prompt on a second model at low.

If the diffs only move when you change models, leave effort on low for that class of work. If high is the one that survived tests, that's your five percent: subtle correctness, races, parsers, "don't change behavior." Use the expensive setting for that shape, not as a lifestyle.

I think most people are overusing effort because the default is the expensive side, the control is labeled like quality, and the feedback saturates immediately. You only notice if you meter it. Almost nobody meters it. The planning and orchestration work I wrote about in The AI-Augmented Developer only compounds if you aren't spending the budget on thinking the model didn't need. Start low. Climb only when the first answer is wrong. Switch models before you treat thinking harder as a substitute for a different set of eyes.

If you're trying to figure out which models and workflows actually pay off in your engineering org versus which ones just burn tokens, that's the kind of question I work on. Get in touch if it's worth talking through.