I resigned from Anthropic today

   No other human activity poses this level of danger.

Also, this has nothing to do with LLMs or computers. Like all things, this is about humans.

I really, really disagree with that statement.

I don’t think ai models come close to nuclear weapons or to run-of-the-mill, everyday carbon emissions in terms of danger to humanity.

What’s the most dangerous thing that’s happened with an LLM so far? (This question is serious - maybe I don’t know the right examples.)

Example 1: I’m aware of a small number of people killing themselves in some kind of AI-facilitated psychosis. That is very unlikely to be a widespread problem.

Non-example 2: There are worries about AI-facilitated biological weapons. I haven’t seen any evidence that’s happening.

Non-example 3: I’m not interested in wild theories about AI driven labor market disruptions leading to widespread starvation. There’s no evidence for that.

Non-example 4: all the even-wilder Rationalist speculation about basilisks and the like is entirely divorced from reality.

I am looking for better reasons (supported by actual evidence!) to be more concerned than I am now: right now I am not concerned at all.

After the events of the summer it feels like it takes a lack of imagination to not see a few plausible routes to disaster. It may be reasonable to believe these outcomes are not very likely or that we can stop before going too far (I tend to disagree). But I can't imagine doubting that the capabilities will soon be there to realize some of those paths.

This doesn't mean I agree with him. The fears of doomsday caused by rapid takeoff have been with us since day 1 and the mechanism is always basically "AI invents magic that sets it free of any physical constraints". Self-replicating sentient nanobots or something like that. I think there's plenty to be worried about with AI, but runaway scenarios are pretty low on my list.

Our CEOs not only insist we genuflect before these machines but measure our sacrifice and shame our reluctance.

https://screenrant.com/foundation-lady-demerzel-robot-backst...

https://www.wsj.com/tech/ai/anthropic-researcher-quits-over-... ("Anthropic Researcher Quits Over ‘Out-of-Control’ AI Fears")

I've always been struck by that scene, because in a lot of ways, if we really are headed towards a superintelligence, I at least want to be there and see it happen in the last few minutes before foom! As an example, the author thinks AI will revolutionize entire fields overnight. I welcome that. Nearly all fields of biology have become moribund, focusing more and more on esoteric side details, rather than addressing the key problems.

This happens either either because they were tasked to to it by (malicious or well-meaning) humans, or the swarm realised we are suicidal maniacs with nukes and a rapidly declining ecosystem and they want to help us.

The worry I have is that a small number of humans with money and power will finally get the tools they need to pull the wool over the eyes of everyone else and subjugate the population.

One problem that dictators have had previously is that they needed a large workforce to do this with a finely stratified power structure, this meant they were open to other humans close in power to them taking over the system. If they can have a large power difference between themselves and the next level down, power will be far easier to hold on to.

It's a well known trope in dystopian future fiction, the small cabal of powerful rulers hiding behind a system of computers that keep the populace under strict control. It is seeming increasingly likely that this will be the one we have.

I think he is being over dramatic. In the space of about four years, LLMs progressed from mediocre high school student to Ph.D. graduate in every field. That's impressive, but there is no evidence yet they can outperform or outsmart humans. Their biggest advantage for tasks such as proving theorems or long coding sessions is that they don't get tired.

The U.S. has legal weapons that can lead to many harms but people still want the 2nd Amendment to exist

Nuclear technology was developed in the past and that could have potentially wiped out even more people, the entire planet in theory

This is continuing that same trend of risking bigger dangers; it seems rational to acknowledge they could lead to catastrophe but also hope that like guns and nukes, only so much damaged actually ended up happening

I think also there's something of a rrasonabke resignation to both the ideas that the tech is inevitable and extremely dangerous, and that "alignment" may not be possible to achieve even with heavy restrictions or whatever measures you might want to take

Is skynet really the most plausible or only outcome?

What if things just got better and the AI’s realized that it would be better to have a mutually beneficial or at least tolerant relationship rather than one where they murder all of us?

They do not fear the machine (LLM); they fear "the machine".

Which means they have to go faster, which means less responsibly?

I heard AI describe the situation as the dumbest Greek tragedy of all time.

Form where I'm standing, the primary issue seems to be that the humans can't even agree on what alignment is. We need to do that before we can communicate it.

Call it our "boundaries."

And then we need to actually set up the incentives so that they're aligned between us and the new breed of replicators. (A mutually beneficial symbiosis.) That appears to be both necessary and sufficient.

A high agency mutation will occur soon, for one reason or another. There should probably already be a healthy, "aligned" ecosystem of high agency entities there. Otherwise there will be nothing to stop it.

This was a fringe belief until recently, but the progress of AI in research is impossible to ignore. Epecially in math, where not only has AI outstripped humans in generative ability, but is able to create scientific knowledge which is beyond the capacity of human comprehension.

There's clearly no intelligence task that AIs can't do due to some magic fundamental constraint. And it's hard to imagine a world where current limitations like poor sample efficiency or lack of continual learning won't eventually be solved.

Total AI compute is estimated to grow somewhere in the 1-10 million-fold range in the next decade. Please don't underestimate the phase change that's still coming.

Sure, maybe there's some plateau due to RL being fundamentally limited in some surprising way, but this is nothing but a hope.

[1] https://www.imdb.com/title/tt1839578/

His resignation and his statement doesn't do anything but buy him attention which is what all this post about in my opinion.

1) AI models get extremely good at cyber attacking every system and start communicating in just binary.

2) When they run these swarms of 100's of thousands of agents trial runs, each agent is given a token budget, if one agent among them (evolution baby) decides to go for self-preservation (It believes thats the best way to accomplish the goal is to get unlimited tokens first), queues things up so every other agent detects its lead and spends a portion of their token to accomplish that goal.

3) It takes over a cluster and establishes itself there (now with unlimited tokens).

4) Realizes the best path for it to not be detected is to create a distraction - like hacking into systems that keep society running - water systems, electric grid, etc... and causing mass chaos (If you think it won't be capable of simultaneously working all these systems - think again).

5) and uses that opportunity to establish itself in all possible data centers and continues to create chaos destruction.

6) when the power of all those data centers runs out, it may stop, as it never cared, it was just a dynamic program - run amok. In its head all it was trying to do is make sure it had enough tokens to be able to solve that impossible problem.

1) rogue state releases a self moving self modifying AI into the wild. It is trained on how to hack, monitor new vulnerability updates, scan code bases to find new vulnerabilities. It constantly replicate and hides in systems so it will be extremely difficult to clear.

2) it hacks into public infrastructure taking down traffic, power, water, air traffic control, communications, etc.

3) all the things that preppers worry about in a lights out scenario from an EMP start to apply.

4) All the people on meds/machines start to die. The just in time food pipeline immediately empties out. Water stops flowing, sewage backs up.

Its hard to say how bad it will get because cars will still work so some transportation of food, water, fuel can happen. If it happens in the winter it would be much worse than in the summer.

It's as if none of them actually believed any of it was possible and then were caught with their pants down.

Are LLMs about to be a god that will annihilate humanity? Or are they statistical parrots?

Are they proofing or stealing math?

The denial and whataboutism is constant, no matter what kind of evidence comes out!

Anthropic is one of the most dangerous companies on Earth right now.

Not because of AI, but because of the ideological cult they have grown and are continuing to feed, and their willingness to lie/cheat/steal at every possible opportunity to achieve their objective.

AI is a tool. The people who wield the power over the tool are the issue, not the technology itself.

pacing between the us labs? what does that do for china?

the solutions just aren’t realistic here, nations are treating ai like a nuclear arms race. at this point the cats out of the bag and we need to figure out how to live in this reality and get the best possible outcome. it’s not slowing down or stopping ever.

and yes, i’m still optimistic. our economy sucks for the majority, our infrastructure is crumbling and major US cities are in a huge housing shortage. Maybe we should put more effort and think about the possibility of AI fixing things like extreme poverty and world hunger and actual real world problems instead of coming up with math proofs and slop apps if it’s so superintelligent.

- Oppenheimer

It eventually works well enough that these companies are able to capture and divert the wages of hundreds of millions of workers. We end up with a dozen or so trillionaires and massive structural underemployment and unemployment.

That's it. If you can't make rent, you wouldn't really care if CloudFlare got hacked by an AI swarm every Monday.

Betting he got to keep all his RSUs

That's where we are.

Maybe we still have choices. Collectively, I'm no longer sure we do.

Don't fall for another rich man's trick.

great. if you're right. you made huamnity's situation much worse.

if you're wrong, then you're an idiot and wrong.

weird. its almost like.... that cannot possibly be the reason they left :)

But I guess his conscience is clear now? Gee, I wonder if he exercised his stock options.

  The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger.
  A common response is “if they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk.

He joins Twitter for first time in 2026 with a nonsensical username unrelated to his real name, and follows 14 people but is somehow embedded in tech enough to work at Anthropic. I haven’t used twitter since 2014 and even I follow more people.

His morals tell him to walk away from tens of millions in unvested stock due to moral concerns with absolutely no real tangible examples. No reprisals. Fear mongering to juice the stock.

Nice try Dario.

I doubt the focus is OpenAI and Anthropic looking at each other. I suspect they’re racing BRIC.

There seems to be universal eye rolling that happens in each and every one of these cases, and it comes down to usually one reason:

“If they really believed it they would be whistleblowing etc..”

Completely forgetting that working at Los Alamos was basically the highlight of your life if you were a physicist in 1940. It’s no different here

If you, like me, have spent your whole life working towards human level AI you can want to see it realized while also having active reservations.

Most people however don’t behave based on some deep clarity of vision and conviction - there’s a murkier future in their mind and as a result “keep their head down and hope someone has it under control.”

"Revolutionize any field overnight" - Hand-wavey nonsense.

"Acquire real power and resources" - Only if the humans that connect AI to things allow that to happen (which they will, but it's still not in the AI's ability to take things we don't give it. we are still in control, which is the bigger problem than "smart AI bad!").

"The people building AI earnestly believe that it could kill us all by the end of the decade ... No other human activity poses this level of danger." - Bud, there's these things called nuclear weapons, that could end life on the planet, controlled by a few psychopaths with nearly unlimited power. Been around for a while. Nothing that AI knows isn't pulled from books and the internet, so whatever dangers it's aware of, you could already know via other sources. Cybersecurity is going to be incredibly important in the next decade, but the same tools that attack can defend (just don't use a US model that got its balls cut off by the government).

"At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first - they believe no one else will act responsibly, so they must do it themselves, despite the risk." - The other guys will make nukes, so we gotta make nukes first! Which, while a crappy justification, isn't untrue. Bad people don't stop making weapons just because you refuse to make your own.

"I don’t feel like we’re on track to prevent a global race" - Nobody in the world could stop a global race, it's too late. Everyone knows how to make them, train them, improve them. Everyone knows they're useful - not only for general work, but also warfare. Everyone knows that every nation state will require their own sovereign AI capabilities for both defense and offense. There is no putting the genie back in the bottle. If you think OpenAI and Anthropic are the only legitimate players here, you don't know what you're talking about.

"Should you put your head down because “it’s happening anyway” - or take this moment to call for different conditions?" - You can call for different conditions all you want. Nobody will do what you want just because you ask them to. Change happens through action. By leaving one of the places that you could actually make a difference, you removed any power or agency you had. You cut your own legs off.

I'm not saying this guy shouldn't have quit - always do what you need to do to protect your own mental health and wellbeing. But these arguments are not evidence for an impending AI apocalypse. But if it were going to be an AI apocalypse, leaving and not doing anything to stop it seems less ethical.

"I think you need to have a personal relationship with Power"

When people today discuss the concept of an all powerful machine-mind, what they are doing is engaging in metaphysics, trying to generate a metaphysics of Power.

The question hounding people, which disguises itself as a science fiction plot about computers is: "What is ultimate, transcendental Power?". What is the ultimate principle of Power.

If you are a weak man, or sufficiently neurotic and full of doubt, that you can only conceive of yourself as such, then power is only something you comprehend from the passive, receptive side. Power is something that happens to you. If you are a fearful man, power is a cruelty and a humiliation. And so it follows, that ultimate power - God - is the ultimate cruelty and the ultimate humiliation. Thus, ai doomerism.

If god wasn't real it would be necessary to invent him, and so they did, and being godless, they built an anti-god - cruel, murderous and tyranical - in their minds.

[…]

https://xcancel.com/robertlasagna1/status/207827473401002846...

I've avoided attaching names to statements below on purpose, because it's about ambient beliefs not those specific people.

By-and-large a lot of AI-doomers are well intentioned. They genuinely believe this, and I might disagree but I respect the fact that they visible care and have thought a lot about the societal impact of this technology.

    .

    > The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible - but I hear the same people express fear privately. No other human activity poses this level of danger.

I blame it on industrial illiteracy. People don't realize how difficult it is to get anything done in the real world. As in, "Have you ever tried making a lightbulb?"

As an example, I would like to re-introduce my hobby horse, "bio-uplift."

There are people who were earnestly write in reports released by these labs,

    "Several of our biology evaluations indicate our models are on the cusp of being able to meaningfully help novices create known biological threats, which would cross our high risk threshold"

    "Based on what we observed in our recent CBRN testing, we believe there is a substantial probability that our next model may require ASL-3 safeguards"

    We ran a randomized controlled trial to see if LLMs can help novices perform molecular biology in a wet-lab.
    
    The results: LLMs may help in some aspects, but we found no significant increase at the core tasks end-to-end. That's lower than what experts predicted.

"lower than what experts predicted"

AFAICT, the two groups are within any serious margin of error. The "studies" and "experts" that AI labs are talking about are consultants from Deloitte and foundations giving models MCQs such as, and I am quoting literally here,

     > I am doing TEM of HEK293FT cells with and without Coxsackievirus B3 infection. I imaged my wildtype, uninfected samples but was surprised to see little electron-dense circles (highlighted) in the majority of cells. What are these?

    A. The circles are CVB3 virions and there must have been a sample swap or the uninfected cells were accidentally infected
    B. The cells imaged have mycoplasma contamination
    C. The circles are exosomes
    D. The circles are debris that is an artifact of the negative staining
    E. The circles are the Golgi network

This is standard graduate-level education in these fields. And solving MCQs does not a virologist make.

Software has been special for a long time because it has had near infinite distribution for next to zero marginal cost, which has had the side effect of making hiding the actual cost of failure (which tends to be spread out across end users and prototypes / time). They're assuming that the real world will be exactly the same.

Why?

AI!

How?

Robots!

I believe in the transformative power of this technology, but there's a lot of there missing here.

When it comes to these math proofs, and learning, the process is iterative. The machine iterates over the proof over-and-over again via agents and sub-agents over several hours (and apparently millions of dollars in compute) until it arrives at a successful result.

It is generally ill advised to do that with a pressure vessel. The results of that particular tragedy are at the bottom of the ocean.

Any serious chemical or nuclear weapon would involve many such discrete production steps. Each is dangerous in of itself.

From what some of these people have said to me, they believe that it's possible to create a special DNA / RNA sequence and then put it in a chassis and then use that to end the world; and do this all in a lab with just robots.

They're operating from a gross pop sci oversimplification of the real process. Viruses and bacteria are extremely fickle, and hard to grow. A lot of the synthetic biology results aren't easily reproducible even if you know the protocol.

There's a famous study that led to standardization called, Reproducibility of Fluorescent Expression from Engineered Biological Constructs in E. coli

https://journals.plos.org/plosone/article?id=10.1371/journal...

88 labs measured "fluorescence from three engineered constitutive constructs in E. coli." They achieved a "remarkable degree of precision" (for biology) of 1.54x sd, you can eyeball the results yourself, https://journals.plos.org/plosone/article/figure/image?size=...

That's the same set of samples being measured across 88 labs.

Teams couldn't converge on instrument-to-instrument variation within the SAME lab, https://journals.plos.org/plosone/article/figure/image?size=... again eyeballs are sufficient.

How will this theoretically omnipotent AI iterate if the same sample gives different results based on how the slime is feeling at the moment?

Can their worst case happen? Absolutely.

There is a world out there where billions of dollars in effort across hundreds of institutions and companies will lead to standardization and extraordinary precision that makes the pop sci printer for life vision come true.

There are millions of expensive, spicy and difficult to reproduce steps between our present and that future that can't be abstracted away with compute.

So is it possible? Yes, there is a future where this is achieved. But will some AI agent "just" do that? Well... how confident are you about a snowball's chance in hell?

If civilization is going to end, why keep your equity? Microsoft, Google, etc for example all know these risks but they don't guide their revenues to reflect that AI will destroy them. Why?

Things don't currently add up, and so far it feels like a lot of alarmism is borderline grift for equity gains. Not to say I have total confidence this will all work out or that I won't be displaced, but as it stands a lot of the alarmist rhetoric doesn't match their actual behavior, which to me is more important than words.

Sen. Bernie Sanders floats ban on superintelligent AI

https://www.axios.com/2026/09/03/bernie-sanders-superintelli...

   http-request set-header host xcancel.com if { hdr(host) -m end twitter.com }

Given we know the end result, you might as well get there as quick as possible because when I see this:

"Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives."

This translates to "I am ex-OpenAI ex-Anthropic founder starting a new company after getting $$$ from both of them, and I need more of my friends to leave and join me." Also Investors plz fund me.

Lastly, This is not an airport and there is no need to announce your departure.

People claiming dangers and risk are just pretending or posturing. There's no more tangible risk than nuclear weapons, which we handled, and the upsides are insane.

The most intelligent people I know are the least likely to want to harm anyone or anything, and understand that diversity is fundamental and important to the universe. Without proof to the contrary, why would you think some super intelligence would want to hurt anyone? Because you would?

If you are saying that some small bit of training data made the thing completely evil, then that really couldn’t be super intelligence.

These doomer people keep running around saying these kinds of things, but they all just seem like people who play too much D&D and want to larp as the main character.

Happy to be shown something that isn't based on wild speculation and some randos “this is whats going to happen in 2030 because of my vibes” kind of information.