The reflex is to reach for the biggest model in the family. Opus, GPT-4o, Gemini 2.5 Pro. Reasoning modes turned on. Extended thinking maxed out. And then a call volume that turns a Series A into a Series A/2.
For a lot of production tasks the small model was already fine.
Where small models win
Empirically, across our own Forge builds and the MUCRIV Council running this company:
- Classification. Sentiment, intent, category. Ten labels. A well-prompted Haiku 4.5 hits high 90s and costs nothing.
- Extraction. Pull structured JSON out of a support ticket, an invoice, a contract clause. Constrained decoding closes most of the gap.
- Routing. Which downstream tool, which agent, which department. Deterministic small-model calls beat a giant "smart" router every time.
- Summarization of your own text. Not novel synthesis — condensation. Small models were built for this.
- Voice-of-the-brand rewriting. Once you've captured your voice in the prompt, a small model applies it faster and cheaper than the flagship.
Where small models still lose
- Open-ended reasoning over unfamiliar data.
- Code generation with subtle constraints (multi-file, existing conventions, obscure API surfaces).
- Multi-hop synthesis across dozens of retrieved documents.
- Anything that used to require a human to think for more than a minute.
For those, spend the tokens. For everything else, you're paying a premium for capability you never activate.
Practical ratio
Our default in a new Forge sprint is 90/10: small model handles the ninety percent of calls that are shape-of-a-task, flagship handles the ten percent of open-ended cases. We ship one shared eval set and route on eval-set-hit-rate, not on a hunch.
The Council running MUCRIV itself is more skewed: six of seven daily agents on Haiku 4.5, and the flagship only for the Monday synthesis where quality moves the needle. Total model spend, all agents combined, under ten dollars a month.
The eval is the moat
What separates a team that ships on the small model from a team that keeps escalating is one thing: a real eval set. Not a vibes check. A labelled fixture with a pass/fail script. When Haiku fails 5% of your tickets and Sonnet fixes them, the eval tells you the tickets to route. When it fails on none, you keep the small model and pocket the difference.
Model choice isn't a moat. Knowing which model, for which call, on which day — that's the moat.