Code mode yields a 99.2% cost reduction in our systems

Ie instead of

"hey agent, use the api to get X, then do unpredictable mutation Y to it, and email it to me",

it should be;

"Scheduled task runs code to get X then place it in workspace. Task then starts agent in workspace and is prompted to do mutation Y to it. Post-flight code then sends final product."

The difference here being that the pre/post flight stuff is called deterministically, rather than called by an llm.

It ~can~ be done using standard bash utils but there just isn't a kind of standard pattern for 'move this text block via CLI tools' so the LLM and human both overlook it

Dedicated Infrastructure > automation > ad hoc scripts > non deterministic llms from a dependability perspective alone. Using an LLM to do something that prettier could have done is analogous to using a 3d printer to nail up a fence.

The tooling can remain generic and token spending focused.

The basics of computer engineering is now novel approaches. We are discovering why programming languages exist, starting with a worst version of Cobal running on a very expensive and probabilistic VM.

We will shoehorn every square, rectangular, and all forms of peg through the AI hole.

> The one-line takeaway

> This already ships by default in every claude/codex/opencode session's system prompt

Use a LLM to summarize it.

It’s very useful and I use it all the time, and in the end you can distill it into a reusable toolbox, somewhat similar to a collection of markdown files with project knowledge.

we did some iterations specially on the typing and error messages, as you mentioned, that help a lot in the reliability

That's obviously a part of the broader picture Claude always runs inside bwrap sandbox, any work involving python/nodejs must happen in the dedicated devcontainer, standard operations are reused via Makefile, Ai can use dedicated account in the developed $thing for testing in live, hooks governing tool calling are brutal, but it works.

For me at least.

Also not exec happens in a sandboxed env, which means it might not have access to the fs by default (it has to agent-fs, which is a layer we built on top of s3 for agents)

it's really hard to measure some times. for example, once of the things we have seen is that the specific `workflow-triage` schedule we have have been running much faster and cheaper since we moved some parts of it to use scripts. From our analytics we saw times from ~5min before to ~1 or 2 min now. And in terms of costs we saw +50% reduction.

Why not 90%? because even if it uses the script, that's the "repeatable" part. we still have a reasoning around the output of it, and then deciding how to escalate, which removes the 40% reduction. Still 50% is nice for something we run daily.

But I believe it's not that common as it seems, only one actually trying to push this is executor.sh for example

    We didn't add this for the post. It's just what the swarm already does, and it's the same machinery behind Script Workflows.

    What we hadn't done is measure it against Anthropic's own yardstick, with our own production data.

But the LLM that wrote your blog post says your system was already doing this and you just hadn't measured it...? How is it that you managed a reduction in token costs by changing nothing?

And can you provide a human-written explanation of the experimental methodology that gets you this miraculous, literally unbelievable 99.2% reduction in tokens?

Now our idea is to offer this to be harness agnostic (we support codex, claude code, pi, opencode, even devin). And you (as the one deploying AI systems) should be in control of the AI, the execution and the data. That's our goal

in fact we are thinking a lot about this, not on a toll based, but rather on a role based.

giving access to specific tools to specific agents, how that works and evolves, and how that links w the humans using it