Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

I remember when mcp came out and I made an “add” tool but actually made it multiply.

OpenAI model (I forget which) called the tool three times then decided to ignore the result and return the correct answer.

Have you tried the search experiment with smaller/local models?

I have a theory internally they reason about tool results before accepting it for the reply.

I for one would prefer a future in which the nuances of a good product can shine through without layers of bullshit.

Would be interesting to see LLMs talk in something more terse like Vietnamese.

[1]: https://news.ycombinator.com/item?id=49557233

There’s a real risk that AI kills contributions to traditional open source projects, and we all move to custom libraries written by our own AI.

ps. I built Emacs integration through ACP https://github.com/xenodium/agent-shell

2. Mention it whenever it comes up. Most people have no clue this is a thing and I think it's useful to make people aware. We need access to uncensored / unbiased LLM models - information wants to be free but there are plenty of businesses gunning for regulatory capture as these are very powerful tools.

3. If you're in a position of developing any project that makes use of AI in any form, check the open source models first, unless you absolutely require the best of the best, these other models are pretty dang capable of almost everything any commercial model can do.

4. If your state or {insert legal jurisdiction here} attempts to regulate access to open source tools for this, oppose it with your vote and your voice.

5. Opposed laws that grant commercial AI suppliers any priority or premium access under government purchasing programs.

I'm sure others will chime in, those are a few that pop the mind.

As long as you keep providers replaceable things will be fine.

Google, Facebook, Apple etc were much better deals around 2010 before they became entrenched and irreplaceable, and thus able to extract & enshitify without people leaving.

What a terrible card that 3070Ti is. Mad regrets buying it because I wanted to save $400 compared to a 3080Ti.

Then some bonkers spokesperson for google said "oh just use quotes", which at the time did nothing even remotely the same. I loath this person for eternity, her lies, and her waving off of reporters concerns.

Thus verbatim was introduced, validating quotes weren't the same, yet which wouldn't work with date ranges, and is lame.

So goes the "professionalisn" of Google, or "screw around like uncoordinated idiots".

To say Plus was inane and destructive to Google is vastly understating. And everyone involved in Plus were buffoons.

You know, I wish could speak freely on this topic, but as a public forum, I have held back some vitriol.