A 4B Model Beat OpenAI's Flagship for $0.00092 a Query
Here's the number that should make every AI infra team pause: $0.00092. That's what Neon and Castform say it costs to run a complex retrieval query on their tuned 4-billion-parameter model. The same query on OpenAI's GPT-5.6 Sol? $0.0873. Do that math at scale and 100,000 queries a day runs you $92 versus $8,700.
That's not a rounding error. That's a company-ending cost difference if you're building a search product on frontier-model economics.
The setup, published in a joint blog post on August 5, is almost embarrassingly simple. Take a 4B open-weight model from Alibaba's Qwen3.5 family. Post-train it with Castform's RL pipeline on synthetic multi-hop Q&A pairs generated from a target corpus, with grounding filters to keep the training data honest. Run the whole retrieval loop on Neon's Postgres stack using pg_search and pgvector. Benchmark it against Sol.
The result: 1.447 for the tuned small model versus 1.369 for Sol, on whatever internal retrieval metric they're using. A win, by their own scoreboard, for a model that's a rounding error in size compared to a frontier system.
The pricing gap is the real story here, and it's not subtle:
- GPT-5.6 Sol: $5 per million input tokens, $30 per million output tokens
- Tuned open models (hosted): roughly $0.03 input, $0.15 per million output tokens
That's not a 2x or 5x difference. That's the kind of gap that makes CFOs start asking uncomfortable questions about your model routing decisions.
What Nobody Is Talking About
Everyone's fixated on the 100x number, but almost nobody in the HN thread — and there are 77 comments and 313 points worth of nobody — is interrogating the benchmark itself. This is a vendor-authored comparison. Neon sells Postgres infrastructure. Castform sells RL post-training. Both companies have every incentive to make this story look as dramatic as possible, and the methodology details that would let you actually reproduce their claim are thin.
<> The biggest criticism is methodological: the headline cost reduction comes from a company-authored benchmark, so the full evaluation protocol, workload mix, and reproducibility constraints matter a lot./>
That's not cynicism for its own sake. It's the same pattern every efficiency claim in this industry follows: startup builds narrow benchmark, startup wins narrow benchmark, startup writes blog post declaring the death of frontier models. We've seen this movie with distillation claims, with quantization claims, with every "we beat GPT-4 with a 7B model" post since 2023. Sometimes the claims hold up under scrutiny. Often they quietly don't.
What's actually true, and matters regardless of methodology gripes:
1. Small models tuned on narrow tasks with good synthetic data genuinely can compete on specific workflows — this isn't new, it's just newly cheap to do.
2. Retrieval and agentic search are exactly the kind of narrow, well-defined tasks where this approach works best.
3. The gap doesn't generalize to general intelligence — nobody's claiming a 4B model can replace Sol as your all-purpose assistant.
The honest takeaway isn't "small open models beat frontier models." It's that model selection is now an engineering decision, not a brand loyalty test. If your workload is narrow enough to build a good dataset for, you probably shouldn't be paying frontier prices for it. That's a genuinely useful lesson, buried under a genuinely oversold headline.
