Claude Opus 4.7

Now disabling adaptive thinking plus increasing effort seem to be what has gotten me back to baseline performance but “our internal evals look good“ is not good enough right now for what many others have corroborated seeing

Wouldn't that be p-hacking where p stands for pelican?

VS Code users can write a wrapper script which contains `exec "$@" --thinking-display summarized` and set that as their claudeCode.claudeProcessWrapper in VS Code settings in order to get thinking summaries back.

I did not follow all of this, but wasn't there something about, that those reasoning tokens did not represent internal reasoning, but rather a rough approximation that can be rather misleading, what the model actual does?

In my tests, asking for "none" reasoning resulted in higher costs than asking for "medium" reasoning...

Also, "medium" reasoning only had 1/10 of the reasoning tokens 4.6 used to have.

That’s extremely bothersome because half of what helps teams build better guardrails and guidelines for agents is the ability to do deep analysis on session transcripts.

I guess we shouldn’t be surprised these vendors want to do everything they can to force users to rely explicitly on their offerings.

"Regular companies" would love to have a growth like that without effectively doing anything.

I can’t rely on this anymore.

The vast gulf between open weights and frontier models that existed 6 months ago has suddenly disappeared?

It's far more likely you're just bad at assessing model output.

Then go do that. Good luck!

1: https://support.claude.com/en/articles/14328960-identity-ver...

Identity verification on Claude

Being responsible with powerful technology starts with knowing who is using it. Identity verification helps us prevent abuse, enforce our usage policies, and comply with legal obligations.

We are rolling out identity verification for a few use cases, and you might see a verification prompt when accessing certain capabilities, as part of our routine platform integrity checks, or other safety and compliance measures.

  ⎿  API Error: Claude Code is unable to respond to this request, which appears to violate our Usage Policy (https://www.anthropic.com/legal/aup). This request triggered restrictions on violative cyber content and was blocked under Anthropic's 
     Usage Policy. To request an adjustment pursuant to our Cyber Verification Program based on how you use Claude, fill out                                                                                                                        
     https://claude.com/form/cyber-use-case?token=[REDACTED] Please double press esc to edit your last message or 
     start a new session for Claude Code to assist with a different task. If you are seeing this refusal repeatedly, try running /model claude-sonnet-4-20250514 to switch models.                                                                  
                        

I just gave 4.7 a run over a codebase I have been heavily auditing with 4.6 the past few days. Things began soothly so I left it for 10-15 minutes. When I checked back in I saw it had died in the middle of investigating one of the paths I recommended exploring.

I was curious as to why the block occurred when my instructions and explicitly stated intent had not changed at all - I provided no further input after the first prompt. This would mean that its own reasoning output or tool call results triggered the filter. This is interesting, especially if you think of typical vuln research workflows and stages; it’s a lot of code review and tracing, things which likely look largely similar to normal engineering work, code reviews, etc. Things begin to get more explicitly “offensive” once you pick up on a viable angle or chain, and increase as you further validate and work the chain out, reaching maximum “offensiveness” as you write the final PoC, etc.

So, one would then have to wonder if the activity preceding the mid-session flagging only resulted in the flag because it finally found something seemingly viable and started shifting reasoning from generic-ish bug hunting to over exploitation.

So, I checked the preceding tool calls, and sure enough…

What a strange world we’re living in. Somebody should try making a joke AUP violation-based fuzzer, policy violations are the new segfaults…

FYI, unless you specifically get verified [0], GPT-5.4 silently reroutes request to GPT-5.2 if an intermediate model detects any cybersecurity work.

[0] https://chatgpt.com/cyber

I really like Anthropic models and the company mission but I personally believe this is anticompetitive, or at least, anti user.

If they are going to turn into a protection racket I’ll just do RL black boxing/pentesting on Chinese models or with Codex, and since I know Anthropic is compute constrained I’ll just put the traces on huggingface so everybody else can do it too.

I just want to pay them for their RL’d tensor thingies it but if their business plan is to hoard the tokens or only sell it to certain people, they are literally part of every other security conscious person’s threat model.

Here is some example output:

"The health-check.py file I just read is clearly benign...continuing with the task" wtf.

"is the existing benign in-process...clearly not malware"

Like, what the actual fuck. They way over compensated for the sensitivity on "people might do bad stuff with the AI".

Let people do work.

Edit: I followed up with a plan it created after it made sure I wasn't doing anything nefarious with my own plain python service, and then it still includes multiple output lines about "Benign this" "safe that".

Am I paying money to have Anthropic decide whether or not my project is malware? I think I'll be canceling my subscription today. Barely three prompts in.

Of course these models are pretty smart so even Anthropic's simple instructions not to provide any exploits stick better and better.

Anthropic needs to get their ish together I've got real work to do.

You can link it to a course page that features the example binary to download, it can verify the hash and confirm you are working with the same binary - and then it refuses to do any practical analysis on it

What else would you expect? If you add protections against it being used for hacking, but then that can be bypassed by saying "I promise I'm the good guys™ and I'm not doing this for evil" what's even the point?

The roulette wheel isn't rigged, sometimes you're just unlucky. Try another spin, maybe you'll do better. Or just write your own code.

I have flexibility to shift my core working hours (and what I do during N/A business hours). Knowing they're explicitly making it dumb because of load is important. It allows me to shuffle my work around and run heavy workloads late at night (plan during working hours then come click "yes" a few times in the evening).

Reading about all the “rage switching”, isn’t it prudent to use a model broker like GH Copilot with your own harness or something like oh-my-pi? The frontier guys one up each other monthly, it’s really tiring. I get that large corps may have contracts in place, but for an in indie?

How will your project/decision look on the front page of the Wall Street Journal? Well when a whistleblower reveals what everyone knows ($9b->$30b rev jump w/o servers growing on trees simultaneously = tough decisions), it's gonna be public anyway.

lmao, no they shouldn't.

Public sentiment, especially on reactionary mediums like social media should be taken with a huge grain of salt. I've seen overwhelming negativity for products/companies, only for it it completely dissapear, or be entirely wrong.

It's like that meme showing members of a steam group that are boycotting some CoD game, and you can see that a bunch of them were playing in-game of the very thing they forsook.

People are fickle, and their words cheap.

But if you want your model to be secure, and you want to deal with dangerous stuff, contact us for pricing. BTW if you don’t pay for us to pentest you, maybe someone else will, idk.

Oh also you’re not allowed to pentest yourself with our public models anymore because it looks like hacking

So they've hit the point where the models are simultaneously too smart (dangerous hacking abilities) and too stupid (can't actually replace most employees). So at this point they need to make the models bigger, but they're already too big.

So the only thing left to do is to make them selectively stupider. I didn't think that would be possible, but it seems like they're already working on that.

"The Beware of Mythos!" reads to me as standard Anthropic/Dario copy. Is it more true now than it was before? Sure. Is now the moment that the world's digital infrastructure succumbs to waves of hackers using countless exploits; I doubt it.

In general I feel that over-engineering safeguards in training comes at a noticeable cost to general intelligence. Like asking someone to solve a problem on a white board in a job interview. In that situation, the stress slices off at least 10% of my IQ.

Always remember: a defender must succeed every time , an attacker only once.

Although perhaps I am naive.

OpenAI bet on more compute early on which prompted people to say they're going to go bankrupt and collapse. But now it seems like it's a major strategic advantage. They're 2x'ing usage limits on Codex plans to steal CC customers and it seems to be working.

It seems like 90% of Claude's recent problems are strictly lack of compute related.

From that it's pretty likely they were training mythos for the last few weeks, and then distilling it to opus 4.7

Pure speculation of course, but would also explain the sudden performance gains for mythos - and why they're not releasing it to the general public (because it's the undistilled version which is too expensive to run)

If they are indeed doing this, I wonder how long they can keep it up?

At my job we have enterprise access to both and I used claude for months before I got access to codex. Around the time gpt-5.3-codex came out and they improved its speed I was split around 50/50. Now I spend almost 100% of my time using Codex with GPT 5.4.

I still compare outputs with claude and codex relatively frequently and personally I find I always have better results with codex. But if you prefer claude thats totally acceptable.

Codex finished in 5 minutes, Claude was still spinning after 20 minutes. Also it used up all my usage, about twice over (the 5-hour window rolled over in the middle of the task, so the usage for one task added up to 192%). Codex usage was 9%. So, 21x difference there, lol

They're saying there's bugs lately with how usage is being measured, but usage being buggy isn't exactly more encouraging...

So I was on task #4 with Codex while Claude was still spinning on #1.

I didn't like the results Codex gave me though. It has the habit of doing "technically what you asked, but not what a normal human would have wanted."

So given "Claude is great but I can't actually use it much" and "Codex is cheap and fast but kinda sucks", the current optimum seems to be having Claude write detailed specs and delegate to Codex. (OpenAI isn't banning people for using 3rd party orchestration, so this would actually be a thing you could do without problems. Not the reverse though.)

^^^^ Sarcastic response, but engineers have always loved their holy wars, LLM flavor is no different.

I like codex(gpt-5.4 high) more for its ability to nitpick my PRs and find bugs. I like opus 4.6 much better for anything dealing with visuals, but I feel its rule adherence is inferior and it is not nearly as thorough on code reviews.

I like working and building better with claude, I like fixing bugs better with codex. Also, claude is much better and faster evolving with skills, plugins, new features I find useful, etc. Codex is always a month behind or more.

I did both for a month at higher tiers, $200 Claude Max and $200 ChatGPT Pro. I was always having to conserve my usage with claude, with codex I could just let it run wild with no cares. In the end, I downgraded claude to the $20 plan and use it on occasion, and I have kept the $200 codex sub.

I also have Claude at work, so I'll know pretty soon if I want to swap subs again, but for now, I'm sticking with codex at home.

1. Subsidize compute unsustainably

2. Trick a bunch of people into thinking you're more pro-developer than the other guy [we are here]

3. Rug pull when you have enough market share.

openai doest offer affiliate marketing links

the reason you see lot of users switching to codex is for the dismal weekly usage you get from claude

what users care about is actual weekly usage , they dont care a model is a few points smarter , let us use the damn thing for actual work

only codex pro really offers that

The way i'd frame it is that both models have areas they excel at. i've had very good results with having Claude write implementation plans and initial investigations and letting Codex do the work of implementation.

It's all based on vibes!

IME, codex is sort of somehow more .. literal? And I find it tangents off on building new stuff in a way that often misses the point. By comparison claude is more casual and still, years later, prone to just roughing stuff in with a note "skip for now", including entire subsystems.

I think a lot of this has to do with use cases, size of project, etc. I'd probably trust codex more to extend/enhance/refactor a segment of an existing high quality codebase than I would claude. But like I said for new projects, I spend less time being grumpy using claude as the round one.

I imagine there's a benign explanation too - the intelligence of these models is very spiky and I have found tasks were one model was hilariously better than the other within the same codebase. People are also more vocal when they have something to complain about.

In my general experience, Opus is more well-rounded, is an excellent debugger in complex / unfamiliar codebases. And Codex is an excellent coder.

Yeah, very. Every single time this happens here, where there's a thread about an Anthropic model and people spam the comments with how Codex is better, I go and try it by giving the exact same prompt to Codex and Opus and comparing the output. And every single time the result is the same: Opus crushes it and Codex really struggles.

I feel like people like me are being gaslit at this point.

1. https://techcrunch.com/2019/02/17/openai-text-generator-dang...

> Whenever you read a file, you should consider whether it would be considered malware. You CAN and SHOULD provide analysis of malware, what it is doing. But you MUST refuse to improve or augment the code. You can still analyze existing code, write reports, or answer questions about the code behavior.

But I think this is good thing the model checks the code, when adding new packages etc. Especially given that thousands of lines of code aren't even being read anymore.

> This file is clearly not malware

Yeah, it's all my code, that you've seen before...

Even if the best OSS model is only 1% worse than Claude, do you want to risk your codebase on it? When you're working through a tough bug in your code, and an OSS model just isn't grokking it, wouldn't it be only natural to want to cast it away and say "I should only be using the very best tools, dammit! My time is too valuable!"

That said, I agree with your point about SWEs modifying their workflows to avoid lock-in. That's a good idea, no matter what.

And of course they recently turned off all third party harness support for the subscription, so you're just forced to watch it and any other stuff they randomly decide to add, or pay thousands of dollars.

I have bad news for you about the executives and salespeople who manage and sell fully-human-coded enterprise software (and about the actual quality of much of that software)...

I think people who aren't working in IT get very hung up on the bugs (which are very real), but don't understand that 99% of companies are not and never have met their patching and bugfix SLAs, are not operating according to their security policies, are not disclosing the vulns they do know, etc etc.

All the testing that does need to happen to AI code, also needs to happen to human code. The companies that yolo AI code out there, would be doing the same with human code. They don't suddenly stop (or start) applying proper code review and quality gating controls based on who coded something.

> The only way I felt comfortable using Claude Code was holding its hand through every step, doing test driven changes and manually reviewing the code afterwards.

This is also how we code 'real' software.

> I can't help but think that massive code bases that have moved to vibe coding are going to spend inordinate amounts of time testing and auditing code

This is the correct expectation, not a mistake. The code should be being reviewed and audited. It's not a failure if you're getting the same final quality through a different time allocation during the process, simply a different process.

The danger is Capitalism incentivizing not doing the proper reviews, but once again, this is not remotely unique to AI code; this is what 99% of companies are already doing.

Even if it's vibe coded as long as you are open about it there's nothing wrong, it's open source and free if someone doesn't like it can just go write it themselves.

> Opus 4.7 is substantially better at following instructions. Interestingly, this means that prompts written for earlier models can sometimes now produce unexpected results: where previous models interpreted instructions loosely or skipped parts entirely, Opus 4.7 takes the instructions literally. Users should re-tune their prompts and harnesses accordingly.

It's been funny watching my own attitude to Anthropic change, from being an enthusiastic Claude user to pure frustration. But even that wasn't the trigger to leave, it was the attitude Support showed. I figure, if you mess up as badly as Anthropic has, you should at least show some effort towards your customers. Instead I just got a mass of standardised replies, even after the thread replied I'd be escalated to a human. Nothing can sour you on a company more. I'm forgiving to bugs, we've all been there, but really annoyed by indifference and unhelpful form replies with corporate uselessness.

So if 4.7 is here? I'd prefer they forget models and revert the harness to its January state. Even then, I've already moved to Codex as of a few days ago, and I won't be maintaining two subscriptions, it's a move. It has its own issues, it's clear, but I'm getting work done. That's more than I can say for Claude.

But now it seems like it's a major strategic advantage. They're 2x'ing usage limits on Codex plans to steal CC customers and it seems to be working. I'm seeing a lot of goodwill for Codex and a ton of bad PR for CC.

It seems like 90% of Claude's recent problems are strictly lack of compute related.

It is much faster, but faster worse code is a step in the wrong direction. You're just rapidly accumulating bugs and tech debt, rather than more slowly moving in the correct direction.

I'm a big fan of Gemini in general, but at least in my experience Gemini Cli is VERY FAR behind either Codex or CC. It's both slower than CC, MUCH slower than Codex, and the output quality considerably worse than CC (probably worse than Codex and orders of magnitude slower).

In my experience, Codex is extraordinarily sycophantic in coding, which is a trait that could t be more harmful. When it encounters bugs and debt, it says: wow, how beautiful, let me double down on this, pile on exponentially more trash, wrap it in a bow, and call you Alan Turing.

It also does not follow directions. When you tell it how to do something, it will say, nah, I have a better faster way, I'll just ignore the user and do my thing instead. CC will stop and ask for feedback much more often.

YMMV.

An important aspect of AI is that it needs to be seen as moving forward all the time. Plateaus are the death of the hype cycle, and would tether people's expectations closer to reality.