Claude Opus 4.7 Model Card

Posted by adocomplete 1 day ago

Counter175Comment81OpenOriginal

Comments

Comment by bachittle 23 hours ago

So Opus 4.7 is measurably worse at long-context retrieval compared to Opus 4.6. Opus 4.6 scores 91.9% and Opus 4.7 scores 59.2%. At least they're transparent about the model degradation. They traded long-context retrieval for better software engineering and math scores.

Comment by film42 22 hours ago

To be honest, I think it's just a more honest score of what Opus 4.6 actually was. Once contexts get sufficiently large, Opus develops pretty bad short term memory loss.

Comment by tomaskafka 18 hours ago

You can support very long context windows if you don’t mind abysmal recall rate.

Comment by freedomben 22 hours ago

Agreed, I appreciate the transparency (and Anthropic isn't normally very transparent). It's also great to know because I will change how I approach long contexts knowing it struggles more with them.

Comment by RobinL 22 hours ago

Could this be because they've found the 1m context uneconomical (ie costs too much to serve, or burns through users quota too quickly causing complaints), and so they're no longer targeting it as a goal

Comment by Someone1234 21 hours ago

Opus 4.7 is also worse at 256K context. Go look at page 195 and page 196. It is across the board regression, not just 1M context.

Comment by RobinL 18 hours ago

Thanks, interesting. Does this make it more surprising that the other benchmarks have improved? I'm not sure I understand the benchmarks well enough - but I'm wondering whether with agentic workflows it's possible to get away with a smaller more focussed context (and hence lower cost) whilst achieving the same or better performance, because of agentic model's ability to decide what the put in context as they work

Comment by timvb 17 hours ago

what's all this mean in real world use?

Comment by jzig 22 hours ago

At what point along the 1M window does context become "long" enough that this degradation occurs?

Comment by daemonologist 22 hours ago

The benchmark GP mentioned is measuring at 128k-256k context (there's another at 524k-1024k, where 4.6 scored 78.3% and 4.7 scored 32.2%).

The longer the context the worse the performance; there isn't really a qualitative step change in capability (if there is imo it happens at like 8k-16k tokens, much sooner than is relevant for multi-turn coding tasks - see e.g. this old benchmark https://github.com/adobe-research/NoLiMa ).

Comment by the13 20 hours ago

Be brief. No one wants AI boyfriend users who drone on & on about their day.

Comment by teaearlgraycold 21 hours ago

A year ago it felt like SoTA model developers were not improving so much as moving the dirt around. Maybe we’re in another such rut.

Comment by vessenes 20 hours ago

This is an interesting document, in that it reads like a Claude Mythos model card that was hastily edited to be an Opus 4.7 model card.

I surmise that someone at the top put the Mythos release on hold, and the product team was told "ship this other interim step model instead. quickly."

I wonder if 4.7 will be seen as a net step-up in quality; there are some regressions noted in the document, and it's clearly substantially worse than Mythos, at least according to its own model card. Should be an interesting few months -- if I were at oAI I'd be rushing to get something out that's clearly better, and pressing for weakness here.

Comment by the13 20 hours ago

What makes you think that? "it reads like a Claude Mythos model card that was hastily edited to be an Opus 4.7 model card"

Comment by vessenes 20 hours ago

There are more mentions of Mythos than 4.6. Mythos results are nearly everywhere, and vastly exceed 4.7's capacity in almost every case. There are sections that report only research on Mythos, none on 4.7. E.g. user surveys about how beneficial Mythos is internally at Anthropic.

Comment by barneybooroo 18 hours ago

Yeah, the section expanding on how they evaluated Mythos internally is a bit baffling considering how irrelevant it is.

Comment by kube-system 21 hours ago

> Chemical and biological weapons threat model 2 (CB-2): Novel chemical/biological weapons production capabilities. A model has CB-2 capabilities if it has the ability to significantly help threat actors (for example, moderately resourced expert-backed teams) create/obtain and deploy chemical and/or biological weapons with potential for catastrophic damages far beyond those of past catastrophes such as COVID-19.

That's an interesting choice of benchmark for measuring the risk of "Chemical and biological weapons"

Comment by Aboutplants 21 hours ago

Gotta prime those Government fears!

Comment by koehr 23 hours ago

This reads more like an advertisement for Mythos, on the first glance

Comment by Uehreka 21 hours ago

I never understand these critiques. If something is useful and you’re selling it, does that mean any technical document describing its usefulness becomes marketing?

I guess maybe, but then do those documents lose value as technical documents? Not necessarily at all, so I don’t see the point. How are you supposed to describe a useful technical thing to users?

Comment by parsimo2010 20 hours ago

This is supposedly the Opus 4.7 model card. It's okay for it to be marketing for Opus 4.7 and describe what it can do, and even okay for it to talk about what it does better than the last generation. GP was saying it sounds like marketing for Mythos (a different and unreleased model). I don't want the Opus 4.7 model card to be advertising for something else.

For context, the word "Mythos" appears 331 times in a 221 page document. "Opus 4.6" appears 240 times, so a reference to a model that nobody has really used happens more often than the reference to the last generation model.

Comment by ModernMech 22 hours ago

That's why I don't like these "model cards" being presented as if they are some sort of technical document -- they're marketing materials.

Comment by Symmetry 23 hours ago

> The technical error that caused accidental chain-of-thought supervision in some prior models (including Mythos Preview) was also present during the training of Claude Opus 4.7, affecting 7.8% of episodes.

>_>

Comment by 100ms 23 hours ago

    $ pbpaste | wc -w 
    62508
    $ pbpaste | grep -oi mythos|wc -w
    331
    $ pbpaste | grep -oi opus|wc -w
    809

Comment by aliljet 23 hours ago

Have they effectively communicated what a 20x or 10x Claude subscription actually means? And with Claude 4.7 increasing usage by 1.35x does that mean a 20x plan is now really a 13x plan (no token increase on the subscription) or a 27x plan (more tokens given to compensate for more computer cost) relative to Claude Opus 4.6?

Comment by computomatic 23 hours ago

They have communicated it as 5x is 5 x Pro, and 20x is 20 x Pro (I haven’t looked lately so not sure if that’s changed).

They have also repeatedly communicated that the base unit (Pro allotment) is subject to change and does change often.

As far as I can tell, that implies there is no guarantee that those subscriptions get some specific number of tokens per unit of time. It’s not a claim they make.

Comment by msikora 15 hours ago

I think as far as the maybe more important weekly allotment Max 5 is 10x Pro and Max 20 is 20x Pro. For the 5 hour window it is as the names would suggest though.

Comment by DonsDiscountGas 22 hours ago

Definitely 13x, at least for now

Comment by ModernMech 21 hours ago

Feels like buying toilet paper.

Comment by 23 hours ago

Comment by joeumn 23 hours ago

I'm actually surprised at how it performed compared to 4.6 and also compared to mythos. Will be fun to use.

Comment by bicepjai 1 day ago

This card is a 272 page report. So now we are redefining names :)

Comment by albert_e 1 day ago

Does the model card fit in the model's context :)

Comment by anonyfox 21 hours ago

well it will saturate your 5h limit window at least

Comment by STRiDEX 23 hours ago

Dumb question but why are chemical weapons always addressed as a risk with llms? Is the idea that they contain how to make chemical weapons or that they would guide someone on how?

Would there not already be websites that contain that information? How is an llm different, i guess, from some sort of anarchist cookbook thing.

Comment by Philpax 23 hours ago

Both. There's the risk of them instructing a user on how to produce a known formulation (the Anarchist Cookbook solution, as you say), which is irritating but not that problematic.

The bigger issue is that they are potentially capable of producing novel formulations capable of producing harm, and guiding someone through this process. That is, consider a world in which someone with malicious desires has access to a model as capable at chemistry / biology as Mythos is at offensive cybersecurity abilities.

This is obviously limited by the fact that the models don't operate in the physical world, but there's plenty of written material out there.

Comment by rogerrogerr 23 hours ago

The world has been blessed by two connected things:

1. Smart people have economic opportunities that align them away from being evil

2. People who are evil tend not to be smart.

We're breaking both of these assumptions.

Comment by chrisweekly 22 hours ago

"Smart people have economic opportunities that align them away from being evil"

For some definition of evil, some of the time, ok. But as economic opportunities compound (looking at the behavior of the ultra-rich), it seems there's at least strong correlation in the other direction, if not full-on "root of all evil" causation.

Comment by rogerrogerr 22 hours ago

Sure, but that’s not “slaughter a stadium of people with drones” evil or “poison the water supply” evil or “take out unprotected electrical substations” evil.

So much infrastructure is very soft because the evil people aren’t smart enough to conceive of or conduct an attack.

Comment by fwip 21 hours ago

I think you might find that, if you reconsider who the 'evil' people are, you might find that we're already doing that sort of thing.

Comment by Jensson 14 hours ago

Its not capitalists doing that though, its politicians, and politicians in non capitalist countries tend to be more evil.

Comment by fwip 11 hours ago

Correct me if I'm wrong, but there aren't any non-capitalist countries currently waging war on others.

Comment by ben_w 4 hours ago

Capitalism is a continuum, not a binary, hence occasional discussion "China is communist!" "No, it is state-capitalism!"

Is Russia currently capitalist, or non-capitalist? Which is Myanmar?

Anyway, personally I think it's the wrong axis; while capitalism and democracy and free press are often correlated, I think that the latter two are the important ones for actually choosing the lesser evils, though capitalism does generate more options to choose between.

Comment by JohnMakin 16 hours ago

> 1. Smart people have economic opportunities that align them away from being evil

for now

Comment by Der_Einzige 23 hours ago

Good. This is how we will force the world to reckon with the isolated, the disgruntled, and "lone wolf" terrorist. Real "sigma males" actually exist, and when they decide "society has to pay" we are all worse off for it. If Ted Kaczynski (quintessential example of a real actual sigma) had been in his prime operating right now, he'd have mail-bombed NeurIPS and ICLR already. I'm not cool with being in crowds of AI professionals right now for physical security reasons given the extreme anti-AI sentiment that exists from nearly everyone outside of the valley: https://jonready.com/blog/posts/everyone-in-seattle-hates-ai...

Comment by malcolmgreaves 22 hours ago

That’s not quite true. Take a look at all the billionaires destroying society. Being evil is the surest way to get to get rich. In fact it’s the only way to amass that level of capital: there’s no ethical billionaire.

Comment by mikek 21 hours ago

This feels like a wild overgeneralization. People can become rich without resorting to evil methods, especially now with global markets and software. Case in point: Minecraft was wildly successful, and now Notch is a billionaire.

Comment by hxugufjfjf 21 hours ago

Eeeeh not the best example maybe?

Comment by orneryostrich 21 hours ago

Pre-wealth, Notch was friendly, kind, and downright jolly! Even as he started to accumulate wealth, he was donating huge sums of money to various indie games. Whenever a Humble Bundle dropped he would top the leaderboard for the amount he paid for the games. Things took a major turn for the worse after the acquisition and after he left Mojang. That's when he ran out of purpose and turned to drugs and conspiracy theories.

Comment by dcre 22 hours ago

LLMs can tell you exactly how to acquire the materials and manufacture the materials. They might even come up with novel formulations that rely on substances that are easier to get. There might be information about this stuff online but LLMs are much better than random idiots at adapting that information to their actual situation.

On top of LLMs reducing the cost/difficulty, the other reason biological and chemical weapons are such a worry is their asymmetric character — they are much much easier and cheaper to produce and deploy than they are to defend against.

Comment by Aboutplants 21 hours ago

It’s marketing, Fear is one of the most effective marketing tools. That and purpose of government attention

Comment by somesortofthing 20 hours ago

They contain broad overviews(throw some disease-causing bacteria in a sort of rainbow arrangement of increasingly more effective antibiotics, you'll usually get something that's at least very deadly even if it doesn't have pandemic potential) but executing in a real lab takes a ton of trial and error to figure out the details. The issue is that the details ~all exist somewhere in the training dataset already, discovered and documented over the course of unrelated, benign biology research. Ability to quickly and accurately search over that corpus translates to large speedups in the physical development process.

Comment by Nicook 19 hours ago

Probably also a bit of liability. After all its been trained on a dataset that includes a long running joke of trying to trick people on the internet to unknowingly create chlorine gas.

Comment by rgbrenner 23 hours ago

In the same way that all coding docs are available publicly

Comment by CodingJeebus 23 hours ago

WAG but I wonder if a hijacked LLM could also assist with figuring out how to obtain required materials, not just provide the recipe.

Comment by jmward01 1 day ago

Haiku not getting an update is becoming telling. I suspect we are reaching a point where the low end models are cannibalizing high end and that isn't going to stop. How will these companies make money in a few years when even the smallest models are amazing?

Comment by blixt 23 hours ago

Isn't it pretty common for the smaller models to release a little while after the bigger ones, for all the big model providers?

Comment by jmward01 23 hours ago

The last update for Haiku was in October, or in startup land, 10 years ago.

Comment by mvkel 23 hours ago

It seems to be a rule that older models are more expensive than newer ones. The low end models have higher $CPT and worse output. I wonder if the move is to just have one model and quantize if you hit compute constraints

Comment by deaux 22 hours ago

> It seems to be a rule that older models are more expensive than newer ones.

It isn't. Gemini has gotten more expensive with each release. Anthropic has stayed pretty similar over time, no? When is the last time OpenAI dropped API prices? OpenAI started very high because they were the first, so there was a ton of low hanging fruit and there was much room to drop.

Comment by mvkel 19 hours ago

I'm talking about gross margins, not revenue.

It's well known that GPT-4 is much more expensive to operate than the GPT-5 family.

Of course they won't drop the prices; it's pure profit if they make models more efficient.

Comment by qingcharles 19 hours ago

Google is putting a lot of research into small models. Most of my AI budget is now going to small models because I am doing lots of tiny tasks that the small models do great with. I would think a decent chunk of Goog's API revenue probably comes from their small models.

Comment by dkhenry 23 hours ago

The Gemma models are at this point. A 31B model that can fit on a consumer card is as good as Sonnet 4.5. I haven't put it through as much on the coding front or tool calling as I have the Claude or GPT models, but for text processing it is on par with the frontier models.

Comment by make3 23 hours ago

absolutely not on par you're smoking

Comment by dkhenry 22 hours ago

You make a compelling argument, but thankfully I have data to back up my anecdotal experience

This comparison shows them neck and neck https://benchlm.ai/compare/claude-sonnet-4-5-vs-gemma-4-31b

As Does this one https://llm-stats.com/models/compare/claude-sonnet-4-6-vs-ge...

And the pelican benchmark even shows them pretty close https://simonwillison.net/2026/Apr/2/gemma-4/ https://simonwillison.net/2025/Sep/29/claude-sonnet-4-5/

Also this isn't a fringe statement, you can see most people who have done an evaluation agree with me

Comment by jmward01 21 hours ago

I think one area I find hard to get around is context length. Everything self hosted is so limited on length that it is marginal to use. Additionally I think that the tools (like claude code) are clearly in the training mix for Anthropic's models so they seem to get a boost over other models pushed into that environment. That being said, open source and local inference is -really- good and only going to get better. There is no doubt that the current frontier biz model is not sustainable.

Comment by make3 17 hours ago

if you look at the details of the numbers of the benchmarks that you shared, Sonnet 4.5 crushes gemma 4. Somehow the first link doesn't run Sonnet on the multi modal benchmark, that's why the top score looks close, it beats Gemma at every benchmark they actually ran. The arena in the second shows that it actually destroys Gemma 4 as well, not close

Comment by lostmsu 23 hours ago

Just to be clear, did you notice the parent said 4.5?

Comment by cmorgan31 22 hours ago

They are also on par in a lot of classification tasks. I did have to actually use gemma4 and fine tune it a bit but that is part of the value add.

Comment by make3 17 hours ago

I did, what's your point?

Comment by il-b 23 hours ago

Ironically, the website is down

Comment by msla 21 hours ago

PDF, because it isn't marked.

Comment by marginalia_nu 18 hours ago

It's not 1998 any more. All browsers read PDFs now.

Comment by Rekindle8090 19 hours ago

Can someone please explain the point of these incremental upgrades? Just release one model. Then maybe do a .5. Then do the next version.

What is the justification for .4.5.6.7.8.9 when the difference isn't measurable and it destroys productivity because they test the next increment on the previous one without customer consent?

Comment by nothinkjustai 22 hours ago

How much do you want to bet this is Mythos, and Anthropic released it as Opus to avoid embarrassment after all the hype they whipped up…

Comment by NickNaraghi 22 hours ago

232 pages is bullshit. Longer than the Mythos system card? What are you hiding.

Comment by nullc 20 hours ago

The model card doesn't mention if this revision will continue to make up and fan vicious conspiracy theories like the prior one does.

I've getting a small but steady stream of harassment from mentally ill people who get spun up on crazy conspiracy theories and claude is all too willing to tell them they are ABSOLUTELY RIGHT, encourage them to TAKE ACTION, and telling them that people who disagree are IN ON IT.

The other major AI LLM services will shut down the deflect to be less crazy or shut down conversation entirely, -- but it seems claude doesn't. Anthropic is probably the worst about prattling on about safety but it seems like their concern is mostly centered on insane movie plot threats and less concerned about things with more potential for real harm.

I've complained to anthropic with no response.

Comment by pukaworks 21 hours ago

[dead]

Comment by gignico 19 hours ago

So LLMs are destroying the economy and the environment but at least “catastrophic risk” is still low. Ok then…

Comment by deflator 22 hours ago

Model Welfare? Are they serious about this? Or is it just more hype? I really don't trust anything this company says anymore. "We have a model that is too dangerous to release" is like me saying that I have a billion dollars in gold that nobody is allowed to see but I expect to be able to borrow against it.

Comment by hgoel 15 hours ago

Maybe referring to it as welfare is odd, but these points are important. It isn't a good look to have a model that tends to get into self-deprecating loops like one of Google's older models, it's an even worse look and potential legal liability if your model becomes associated with a suicide. An overly negative chat model would also just be unpleasant to use.

With the weights being mostly opaque, these kinds of evaluations are an important piece of reducing the harm an AI model can cause.

Comment by deflator 1 hour ago

I feel that anthropomorphizing the model is also potentially very harmful. We've seen that in the LLM interactions that end in tragedy. It's the wording that bothers me.