A Jev that can ask Opus for help gives Opus accuracy at 4x lower cost

Posted by theanomaly 6 hours ago

Comments

Comment by xlayn 5 hours ago

If you use Qwen3.8 27B to manage, to orchestrate, do do shallow research, and then make it ask claude you will notice that Qwen can get out of Haiku, Sonnet and Opus almost as good answers... I was researching how good the model asnswers where, I can't get to work with anything but Fable, but Qwen can get good work from any of them. My theory is that if you make Qwen understand you, Qwen can elaborate as much as needed to get any of those models get good results, because it's filling in details. So using Qwen to direct claude also get's you amazing results way way cheaper...

Update: let me elaborate more, I read the HN entry about the team of 5 Qwens being close to fable quality... so I embarked to add to my harness (https://github.com/alainnothere/privibe because we all have one right?) and once I had added the orchestrate tool I had to now measure somehow it it's able to improve quality, so I claude and then Qwen to run a experiment to call my harness passing the model to use and ask them to do something, 3 rounds I think and see what quality came out of it... and as per the grading... qwen and claude agreed that the quality of the response comming out the harness using the orchestrate was VERY CLOSE, with opus and fable providing more "in my opinion you should" which matches my experience on why they are better.

One more update: I can't get copilot models to work decently, it may be my way of working... but I can get Qwen to follow what I try to do... if Qwen get's me he can then fire copilot cli and get amazing results by using copilot... bonus points seems like copilot cli is cheaper with the AIC (cost of the requests) so you can make the copilot subscription longer

Comment by drkleiner 5 hours ago

I do the opposite of this, use Qwen3.8-27B as subagents, Opus 5.5 as orchestrator and planner, never implementer, and it cuts my token usage significantly. So it is using Claude to direct Qwen. Never thought of the other way around, will give this a try soon as possible.

Comment by bluegatty 1 hour ago

Both of those orchestrations will work for sure.

Smart agent to orchestrate cheaper agents for labour.

Dumber agents to be very explicit about asking smarter agents for requests that require deep information.

This is a thing - the SOTA models really light up the more information you can provide them, and humans are too lazy / incapable of doing that in every prompt.

Entirely depends on what you are trying to do.

Comment by jeef_berky 2 hours ago

My goal was/is to have a local model as a "stable" part of the system, or at least a part of it that you have full control over, and have it reach out to the cloud for some Opus. In practice, to get results, I operate as you say.

At this point if I could do anything with zero effort or time, I would get Opus 5.5 to make a grand scheme, handoff to the local model, and give it ability to reach out to Opus again.

Comment by ihuman 5 hours ago

What do you use to get Opus to call Qwen?

Comment by Blueskytech 2 minutes ago

Try this out

https://tysonberry-wogan.com/command-and-control-by-tyson/

Comment by unrented7977 2 hours ago

I just fell Claude "there's an LLM server on this port serving this model, use it as a subagent"

Claude will write some scripts to call the LLM API and use it like any other tool.

I haven't found a pressing need for anything more complex than that. My multi-agent setups tend to be a pile of bespoke scripts per project anyway, an MCP server is only gonna get in the way. But I suppose this is because I self-host the models, if you're using OpenRouter or whatever it may need more complexity than a curl call to an unauthenticated port

Comment by hocusPokeUs 3 hours ago

So I use Google Gemini to plan or review; otherwise same loop so brain dumping how I get Gemini to Qwen/Gemma/Meta's model I forget the codename for; local agent reads the AI Studio log locally via Google Drive where they store the logs by default in a JSON format

So with the gdrive app for my OS it just looks like a local file path. Had the local agent learn to parse it.

I still prompt in AI studio direct as needed; local agents see the log change but cannot push back to Gemini their result.

Few reasons:

AI Studio is available where ever via browser, no VS Code or my usual vim setup to miss. I can switch between projects there as an idea hits and the agent switches up the local repo path and session history.

AI studio can search web, etc, etc already so my agent is less complicated with addons in general

Also using AI Studio between local agent turns has been my "human in the middle" approach; to have an explicit gap in the process and check each turn.

I only have a 24GB 3090 most models also take longer locally and I never hit the free tier limits with Gemini :D

Though the latest Meta model is twice as fast at 30B for me as other similar size models. 1-2 hour code gen and QA of build errors is now 30-50 minutes to parse latest changes from Gemini and get a successful build. The Meta model is pretty great at "fill in the middle" from a well structured plan.

Overall this loop feels like a normal workday pace. I freelance so I have a lot more say over that.

Comment by drkleiner 4 hours ago

Custom MCP with a variety of functions, I have different Qwen profiles with a variety of subagent numbers and cache pools, my default is 192k pool with 64k initial context that can grow 32k at a time up to 128k, allowing 3x64k, 2x96k or 1x128k and 1x64k subagents. Qwen uses Pi harness. Model I use specifically is ukisai/Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF

Comment by dev-in 3 hours ago

Why not using a proxy in claudecode to forward all the claude-* model to anthropic as a standard proxy forwarder and route other models to other providers (openrouter, vllm, ..)? This way Claude's model can get all the context from other models as all in the same harness. To use other models than claude-* you can use agents files.

Ps: as you won't use your proxy to distill or alter claudecode requests to anthropic you should be compliant with anthropic rules.

Comment by cjonas 1 hour ago

you can easily run qwen locally in a pi harness (or similar) and just tell claude code to delegate all implementation work to it. claude will even setup the harness with the local model

Comment by irthomasthomas 4 hours ago

Install the llm cli and tell opus to use it to prompt qwen.

Comment by verdverm 5 hours ago

You can have a setup like this with OpenCode agents, where you can define custom top-level agents, subagents, and assign specific models to each.

https://opencode.ai/docs/agents/

There are a lot of ways to do this if you stop using Big Ai harnesses that make it difficult to use models besides theirs.

Comment by nullstyle 4 hours ago

ohmypi will spawn subagents across different providers out of the box. Opus will spawn subagents running on my local ds4 installation just fine

Comment by koolba 5 hours ago

> My theory is that if you make Qwen understand you, Qwen can elaborate as much as needed to get any of those models get good results, because it's filling in details.

How is this different than just telling the model you’re using to fill in the gaps itself?

If you’re not manually reviewing the intermediate hop, how does that add to the “understanding you”?

Comment by xlayn 5 hours ago

there is no difference... each case may be different, I have a local qwen gpu setup... and is fast... but not as fast as claude. Fable is amazing and if it cost the same as my qwen setup I'll be running fable and never had qwen in first place. But Fable is crazy expensive, Qwen is not as fast but cheaper... Haiku is fucking dumb IN MY HANDS... BUT my local qwen is quick enough to be briefed, get the idea and delegate. I work with Fable mostly delegating to opus/sonnet and it's cheaper than just fable, I can't stand working with opus or the other models.

Why Qwen directing the others? For the things claude refuse, Qwen can make very specific questions and still get an answer without triggering the thing.

Comment by make3 3 hours ago

I think that his idea is that a lot of preparation / setup work is boiler plate / brain dead stuff that can be done by a shallower model.

It's like, find the relevant scientific papers by using Google search and the scientific paper MCPs, summarizing stuff, etc. Then the big model can come in and actually think on the artifacts

Comment by rao-v 4 hours ago

Oh man I’m been playing with little -> big vs big -> little sub-tasking a bunch.

My sense so far is it’s very much about problem domain “taste” and there is no one correct answer (that is token cost efficient).

Does the smartest agent need the full flavor of the context of the problem to make real progress (i.e what’s been tried, how you’ve influenced the approach) or does the smartest agent really need a crisp problem definition and context room to really work.

Comment by make3 3 hours ago

I suspect that a lot of the fudging around with MCPs and tools can be delegated to smaller agents. Astra doesn't need to look at 300 websearch results directly or to extract text out of raw HTML

Comment by ninininino 5 hours ago

Anthropic has a native solution to this which is that you can use `/advisor` to adjust config to enable using Sonnet but set Opus as an advisor or use Haiku and then set Sonnet or Opus as advisor that the weaker model will automatically call when it thinks something warrants it:

https://code.claude.com/docs/en/advisor

It allegedly will automatically judge when to call the stronger model, not sure how much instructions in prompts or AGENTS.md can steer it.

Comment by jimbokun 4 hours ago

Add this to the things I never imagined when thinking about what a world with artificial intelligence might look like.

Comment by nico 4 hours ago

This is very cool. But also, it’s a benchmark/result for a very specific experiment/task: identity resolution

Despite the title, the article is not making claims of, or demonstrating general usage improvements or general coding agent improvements

Comment by theanomaly 3 hours ago

Agreed, though many kinds of tasks are likely to benefit from this pattern, YMMV.

Comment by 1 hour ago

Comment by Fordec 5 hours ago

This sort of stuff has me believing that decision model controlled/prioritized context is the future of context management beyond the raw AGENTS.md file.

Why give all the context to an agent, when the context can also be curated, updated and made concise automatically for each new problem.

Comment by yfontana 3 hours ago

I too feel like context management has a lot of room for improvement, but the problem right now is that dynamic context management will most likely tank your cache hit rate. We need caching that's not just based on prefix-matching for this to make sense.

Comment by jeef_berky 2 hours ago

My bandaid fix for this is having a large number of sessions or "agents" and being ok with them being inactive at times, but they maintain their context. This has to be in some kind of system with appropriately sized "departments" to justify that agent's existence, or to justify it splitting into two, or creating sub-agents, etc. Yes, there is a cost when you cold start an agent after a week, but for now I'd rather pull that whole 400k+ context back without layers and layers of /compact. And then hopefully you keep that agent going for long enough to benefit from any cache-hit discounts.

Not really a good analogy, but it feels like having an image you like but every time you view it, it runs it through another loop of jpg compression. Use it when it's most effective, otherwise let it preserve it's details.

Comment by gunalx 2 hours ago

And as I see it that is impossible with current autoregressive global attention models because each part depends on the one before. Maybe if we can cache the local attention in blocks and only recompute the global parts.

Comment by bigcat12345678 5 hours ago

Yes × 1000... Jev is the classic type of scientific discovery, any human can see it's effectiveness without actually trying it. We were only deprived the discovery because of the incredible noises created by the AGI/ASI ferver....

Comment by yieldcrv 2 hours ago

most people only need if statements

wait till they found out about that nanosecond latency “conditional model”

turns out I’ve been an “RLIF” training expert my whole career

Comment by angry_octet 2 hours ago

It's illustrative that there are even errors in how the humans have formulated the task and documented the errors.

They propose:

Valentino → Female Child

Yet this is obviously a boy name, else it would be Valentina. Is this model going to return confidence values? No, they want decisions. This is an inherent problem in AI with reductive logic for these tasks.

Likewise, dates. Is 17/08/22:

- 20170822 (Chinese write the year first, and in speech and chat they will say '17' not '2017'.)

- 20220817 (Everywhere But North America.)

Is 04/06/2009:

- 20090604 (Non-North American)

- 20090406 (American)

Without knowing the cultural context, the location, the document type, the document author, etc, you could be getting it wrong.

Comment by jwpapi 2 hours ago

Germans write dates like 17.02.2026

It’s an interesting problem. And i have following thoughts to it?

- Are all cases even decidable?

- Shouldn’t one false positive rank higher than a hundred rank negative.

-Shouldnt’t the AI just abstain from decisions to avoid false positives.

- Is it smarter to seperate this in a lot of small sub problems and to define this as a rule set of small yes no decicions, why does this need to be solved agentically?

- You should definitely clean up deterministically first.

I feel like if you really want to solve this problem, you would approach it differently. This seems like a wrong approach and a bad article on top?

Comment by jcims 1 hour ago

>Is this model going to return confidence values? No, they want decisions.

The system doesn't return confidence values because that's not what they are designing. The models, however, do and to me it's interesting that Jev's measure of confidence is pretty well calibrated to its correctness.

>Without knowing the cultural context, the location, the document type, the document author, etc, you could be getting it wrong.

And? There's no perfect process, just ones that asymptotically approach correctness at exponentially increasing cost. They own the system and get to decide what works for them.

Comment by IanCal 2 hours ago

It’s a surname. (Edit you can see this in the first row they use it as an example surname)

Comment by netshade 3 hours ago

Found similar results, but the economics of the models involved have changed so quickly that it becomes so task specific on what the right thing to do was.

With an assemblage of models like Jev + Opus or Jev + Qwen, Jev would often do the right thing when deciding to escalate to another model - I was impressed that it would usually make the right call, especially if you gave it good awareness of what the other model capabilities were. "ESCALATE to opus if you need good vision awareness".

When Haiku 5.5 came out though, for my tasks, the cost differences disappeared almost immediately. Haiku was good enough / fast enough for so many things that introducing a decider layer didn't offer huge wins. Jev's "millisecond" level performance was nice but would get dwarfed by other latencies in my chain ( web browsing ), that it didn't make a measurable difference in my tasks.

Comment by gbrindisi 5 hours ago

Excellent read. I use the same hybrid approach for a lot of security engineering work and it’s working well after initial calibrations, for example for our own harness we rebuilt auto mode exactly like this to judge tool calls and intent.

Comment by morsch 5 hours ago

I had the same idea, I extended opencode-warden to use a local decision model to validate tool inputs and outputs. It seems to work... ok? d1 is fast enough to run locally on an old 1080 GTX. Not sure if I can recommend relying on it.

Comment by jeef_berky 3 hours ago

Where is the Jev that can replace 20 Opus sessions asking each other for help in a coordinated way? I want to shift towards a local model that "asks for help" in an effective way, but it still seems a bit out of reach. I mean, you could build your own 27B or 35B or A3B or bumblebee and spend a week or two carefully forming it into something useful, but it feels like that would be too specific and it would be thrown out when something like qwen4 is released. Maybe not, I don't know. Opus still exists as it is, in a "subsidized" form that's hard to step down from.

Comment by ghempton 2 hours ago

Jev is pretty cool for augmenting llms in a lot of ways. We use it in our own agent orchestrator[1] to help route ideas/prompts to running agent sessions. I find when I have a lot of parallel agents running its hard to reason about which agent I should be talking to and Jev makes that call for me.

[1] https://github.com/codealong/codealong

Comment by boutell 4 hours ago

Ideally, "system one" intelligence is good both at quick reactions and at knowing when to hold off and wait for "system two." Sure enough, that works with Jev.

I don't usually get excited about the high-level, "whoooa man" marketing-speak, but the parallel is striking.

Comment by theptip 2 hours ago

This is the Model Router architecture, from another angle. The problem is always, how does a dumb model know it needs to escalate to a smart model?

Some problems are obviously hard known-unknowns (eg “solve Navier Stokes”). The problem is unknown-unknowns or knowledge gaps, where the model doesn’t know they need help.

Comment by bastawhiz 1 hour ago

It's not a problem you can get right, because you can't know if you need to escalate until after you escalate (and what you escalated to disagrees with what your first choice would have been). But that's the point: how cautious you make the model is the knob that lets your trade cost for correctness.

I don't believe the answer is ever "how certain is the model?" It's "how much is at stake?" Or "how hard of a problem does this appear to be?" It's not about teaching the model to have uncertainty, but rather to be avoidant of things that are sufficiently important to get right.

Comment by theptip 1 hour ago

Yeah, it’s a precision/recall tradeoff. And as such you calibrate by the cost of each FP vs FN.

Comment by mannyv 2 hours ago

What's interesting and annoying is that the results only matter for these specific releases of the models, with the specific parameters used.

That isn't obvious, unless you've been doing ML for a while.

Comment by ezotony 1 hour ago

their cost numbers give you the hard set escalation rate even though they don't list it, (2.94 - 0.06) / 5.39 is ~53% going to Opus vs ~22% on the easy set, same thresholds

Comment by johnshahbazian 3 hours ago

I got very good results from JevK5: https://github.com/allebee/jevk5

It's what I use locally. For my workload, it was right up there with Jev.

Comment by ltsSmitty 5 hours ago

Liked the read. I wish there were some code examples for how jev/Luna were actually done as I try wrapping my head around their new paradigm.

Comment by Jaxkr 5 hours ago

> We tested whether fast, cheap decision models like Jev and GPT-6 Luna

GPT-6 Luna is a small LLM, not a decision model :P

Comment by rowanseymour 5 hours ago

No but it's the only OpenAI model with a decisions API https://developers.openai.com/api/docs/guides/decisions

Comment by paxy 4 hours ago

Small LLM + no reasoning + fixed JSON-only output = decision model

Comment by ax0ar 5 hours ago

There's actually a decision model version of GPT-6 Luna... You are right to be confused though. OpenAI and their never ending naming games. https://openrouter.ai/openai/gpt-6-luna-decisions

Comment by intervolz 5 hours ago

4x lower is so mathematically awkward.

Comment by markcerqueira 4 hours ago

That reads fine to me as a native-ish English speaker. What would read less awkward for you?

Comment by mjuarez 4 hours ago

Like a comment on another thread, 0.25x would be the much better way to say this. It's 25% of the original value. It's also consistent: If 3x means 300% the original value, then 0.25x is 25% of the original value.

If you say 4x lower, that changes the semantics for lower than x vs higher than x, which is absurd.

Comment by jcelerier 3 hours ago

> If you say 4x lower, that changes the semantics for lower than x vs higher than x, which is absurd.

where does it say in the rulebook that human languages have to have perfectly symmetric pairs of words?

Comment by ant6n 5 hours ago

So like, -3x?

Comment by timClicks 5 hours ago

More like 0.25x.

Comment by dangoodmanUT 4 hours ago

We've been doing this for a while. Fine tuning the decision model helps a lot too. The problem is you still need a pretty low confidence threshold (e.g 10%), but often times that reduces the work by 70% with 0% quality loss (in our benchmarks)

Comment by lukethorham 5 hours ago

I wonder if something like this is the undisclosed new approach from Anthropic that presumably has massive improvements by combining multiple models. Leveraging decision models as much as possible seems logical for extra speed and token efficiency.

Comment by couldaidothis 4 hours ago

I think OpenAI and Anthropic have both been doing similar "routing" approaches for a while. Wasn't OpenAI one of the first to have a small model start responding to you for the first several tokens so that you got an instant response while the big model became available?

Similarly, Grok has been routing to Opus internally for a little while now I think.

Comment by 2001zhaozhao 5 hours ago

> undisclosed new approach from Anthropic

Any source on that?

Comment by lukethorham 40 minutes ago

There’s a METR audit document which apparently describes various unreleased models. This is discussed in a YT video by Wes Roth. I have no firsthand knowledge about it.

Comment by pearcoder 5 hours ago

they do have the 'advisor' model setting now which also plays well with this

Comment by blurbleblurble 34 minutes ago

The application creeps me out

Comment by jwpapi 1 hour ago

Isn’t that by nature a deterministic problem?

Comment by hadlock 4 hours ago

One of the most immediate and practical uses for AI for most (all?) businesses is CRM enrichment; this is a great article, thanks for sharing.

Comment by xfalcox 4 hours ago

Been doing this on Discourse for Spam and locale detection, mostly doing d3-turbo with Qwen 3.8 Flash Next Help.

Comment by giancarlostoro 5 hours ago

What about Sonnet? Haiku? Does it need Opus specifically?

Comment by rickypp 5 hours ago

Seems to me that the important factor is that Jev has well-calibrated confidence for this task. Which secondary expensive model it defers to is probably less critical in this technique as long as that model has higher performance than Jev.

Comment by couldaidothis 4 hours ago

Yeah exactly - very interesting that Luna is calibrated so poorly despite working better on easier problems.

Comment by phendrenad2 4 hours ago

Jev is interesting because it adds "muscle memory" to LLMs. I think we're approaching some fusion of systems that approaches the human brain here.

Comment by asar 4 hours ago

i'm so numb to these headlines nowadays... i guess i've been disappointed by how effective these approaches end up being for actual work but i feel like it's easy to produce these headlines with "proof" that doesnt prove anything besides the sandboxed scenario it was run in