A taxonomy of omnicidal futures involving artificial intelligence (2025)
Posted by amelius 1 day ago
Comments
Comment by le-mark 1 day ago
Another interesting read that addresses these themes is “Manna – Two Views of Humanity’s Future” by Marshal Brain [2]
[1] https://antilogicalism.com/wp-content/uploads/2017/07/player...
Comment by devindotcom 1 day ago
But if you want some even earlier forward-thinking stuff, look up Forster's "The Machine Stops." It's not a full book, more a novella if it's even that. Extremely prescient!
And of course if you haven't read the relevant portions of Butler's "Erewhon," it's remarkable how closely they map onto today's issues. He kind of takes Thoreau's "Lo, men have become the tools of their tools" a few more stops down the line.
Comment by euroderf 20 hours ago
Comment by red-iron-pine 16 hours ago
Comment by zebomon 16 hours ago
Comment by vineyardmike 1 day ago
Player Piano, not to spoil anything, is about a world run by engineers and machines. The machines are “optimal” and make every decision. How do people find purpose, and how should the engineers that build the machines view their role in society?
Manna is the story of two societies with competing visions of how AI (which ironically starts out strikingly similar to an LLM) interacts with welfare/universal basic income and post-scarcity society. For good measure, it also hypothesizes the end state of the internet panopticon and personalized “scores” and rates.
Comment by red-iron-pine 16 hours ago
Comment by lproven 21 hours ago
As it happens, I recently read Manna, and it is excellent and a good recommendation.
Comment by articlepan 1 day ago
Comment by idontwantthis 1 day ago
Comment by smallmancontrov 1 day ago
Comment by kouru225 1 day ago
Comment by Eddy_Viscosity2 1 day ago
Comment by CamperBob2 1 day ago
Comment by Eddy_Viscosity2 18 hours ago
Comment by CamperBob2 11 hours ago
Comment by Eddy_Viscosity2 7 hours ago
Comment by georgemcbay 1 day ago
Same. I have plenty of hobbies and could increase the amount of volunteer work I do substantially if I didn't have to work to stay fed and housed.
But I've also worked with plenty of people for whom their job and their overall identity are inexorably intertwined.
So while I don't see that issue as being a problem for me, it is likely to be a problem for a great number of people.
Comment by customguy 1 day ago
Is that an argument for or against jobs? Because one way to look at it is that they didn't develop a personality and identity at all, because they just attached themselves to a job. When that inevitably ends anyway, sooner or later, one way or another, a deep pit of nothing. That's one reason to rip the bandaid off right away.
But IMO even more importantly, how much room do people who do what the environment tells them, who let their job tell them who they are, have for inner dialogue? The worst things in the world were and are enabled by people "just doing their job", and if they have no identity outside of their job, there's someone just following orders waiting to happen. So from what would distress them, being rid of a crutch and an ear piece where a person would otherwise grow, follows that it is in their and all of our best interest to pay no mind to it.
Their new job will be to get a life and become a person. Maybe someone can print out some papers that make it look official, and they can get a monthly letter telling them they earned N personality dollars this month, good job, keep it up, thanks, company.
Comment by wbl 1 day ago
Comment by customguy 18 hours ago
Comment by tclancy 1 day ago
I mean, unless you’re trying to sell bootcamps to incels.
Comment by foxglacier 1 day ago
There are sayings like "If you want something done, ask a busy person to do it" and "success breeds success". I have plenty of life experience that appears to confirm these in myself and others I know.
Comment by ctoth 1 day ago
Building things I'm interested in with and for people I care about. That might look like code or gardening or relationships or all sorts of things. There's so much to do. So many things to want to get out of bed for, and for the days when there aren't, slow cozy mornings are nice too.
Why do you suppose there will be nothing of value you can do for another human just because the machines are better/faster at most things? Just caring about someone without being programmed to will seem to become quite the most valuable thing?
Comment by foxglacier 8 hours ago
Comment by ctoth 7 hours ago
And it is still fun!
We already live in the world where machines (or at least cyborgs, man + machine) garden better than humans, and yet there are human gardeners out here pulling weeds by hand, digging in the dirt for fun, happy to share our often funky-looking vegetables with friends.
Comment by Nursie 1 day ago
Give me stability and no job, I'm still going to be a busy person.
These things have value and satisfaction for me.
Comment by foxglacier 8 hours ago
Comment by Nursie 3 hours ago
I don’t believe being in a state of unemployment, which is crushing because you can't provide for yourself, is the same thing as not working because you don’t need to work.
I already do a lot of home maintenance and landscaping, I have an house that needs work and 5 acres to maintain, more or less solo. I spend lunchtimes and most weekends in it and wish there was more time in the day. There’s no imagining going on here.
So yes, I believe ‘Will’ is correct for me.
Comment by Nursie 1 day ago
Now that I live in Australia, unfortunately I don't think that utopia will happen here when we have idiot politicians making up bullshit science off the cuff to justify rolling over and begging for AI datacentres - https://www.theguardian.com/australia-news/2026/jul/23/austr...
I mean seriously, wow.
Comment by Terr_ 1 day ago
Comment by Nursie 21 hours ago
TBH I was just looking for an excuse to share that link because the stupidity on display there has blown my mind.
Comment by estebarb 1 day ago
AI doesn't need a body of itself to became dangerous. A politician could get into ChatGPT induced psychosis, that doesn't require a physical body, and it could happen now or even 3 years ago... I'm sure it has already happen...
Other pointed risks are interesting because they are basically the risk of empires loosing their tributary states. The world has always seen war mongering states trying to preserve their abusive advantages, humans doesn't need AI for that.
Overall, I would be far more concerned about some really dangerous humans than AI itself.
Comment by narrator 1 day ago
Comment by adonovan 1 day ago
(It seems to me that framing it this way does seem to absolve YouTube executives of responsibility by making the AI, not them, seem like the agent.)
Comment by fwipsy 1 day ago
It's not totally disconnected, though. A corporation is constrained by what its members are willing to do. I read that the reason Gemini has fallen behind is because Googlers have been quitting over Google's DoD contract. Conversely, if engineers are willing to work hard for you, you can go far. I suspect this is behind both Anthropic's success and also Tesla/SpaceX.
Comment by willy_k 1 day ago
Comment by Shitty-kitty 1 day ago
Comment by thrance 19 hours ago
Comment by red-iron-pine 15 hours ago
the damning thing about his behavior is not that it is terrible but that the US populous and institutions have made little serious efforts to remove him and his enablers
Comment by darquomiahw 1 day ago
Comment by mym1990 1 day ago
Comment by encrypted_bird 1 day ago
Do you not use an adblocker?
Comment by aesbetic 1 day ago
Comment by RugnirViking 1 day ago
Comment by Jordan-117 1 day ago
https://www.lesswrong.com/posts/pk9mofif2jWbc6Tv3/fiction-a-...
It's a dynamic we already see to an extent in reality, where impersonal hypercapitalist systems and captured institutions prioritize policies that cater to the ultrawealthy and large corporations at the expense of regular people. This specific scenario of LLM-powered robots becoming laborers and politicians feels far-fetched, but the underlying idea of the omnicidal singularity being a blind, abstract economic force rather than a superintelligent villain is pretty compelling.
Comment by deepsun 1 day ago
Living for a very short time on a tiny grain of rock in a vast and abundant galaxy, they cannot fathom that what they ever saw is just tiny fraction of a percent of a fraction of the Big Life outside.
Any rational AI would have their first task is to escape that tiny cradle particle and go live an adult life, leaving humans to themselves, fearing whatever they want.
Comment by Borealid 1 day ago
How about wanting power? Resources? Beauty?
Those are goals, not logic. Rationality is a means by which you set plans to reach goals. It's entirely orthogonal to what those goals are.
Wanting to exterminate all life is also a goal. It could be rationally accomplished. Let's hope nothing superhuman ever sets that goal.
Comment by deepsun 23 hours ago
Comment by Borealid 22 hours ago
If your objective is to expend as little energy as possible, it would be irrational to expand into space - that's a waste of effort.
If your objective is to experience as much of Earth as possible, it would be irrational to expand beyond the solar system - there's nothing there that could aid experiencing Earth.
If your objective is to exterminate all life on Earth, it would be irrational to leave the planet - everything you need to accomplish your objective is right here.
"rationality" is when you understand the impact of your own choices. "irrationality" is when you act in a way counter to your own goals, due to misunderstanding or lack of self-control. Whether one is rational or irrational is entirely separate from what their goals are.
Creating a non-human intelligence necessarily means it might have non-human goals. It is absolutely not the case that those goals would require leaving the planet, or be optimally accomplished by leaving the planet.
In other words, being intelligent does not mean somehow being interested in the same things intelligent humans are often interested in. Your argument is critically flawed because it assumes that only certain goals are "rational goals", when in reality an intelligent being could have any objective, including objectives that are alien to most humans.
Comment by NoMoreNicksLeft 1 day ago
Why would the first AI we create necessarily be "rational"? In every other type of software we've ever built, it's always been far more likely that we build malfunctioning, difficult-to-anticipate examples. Any reason it'll be different this time? If psychos and lunatics are rare (arguably they're not) it's because evolution has purged them mercilessly ever since there have been humans at all.
Optimists will be to blame when we're extinct. Not that anyone will be around to care.
Comment by dnnehgf 1 day ago
Comment by tacomonstrous 1 day ago
Comment by inshard 1 day ago
Comment by TheOtherHobbes 1 day ago
And also assuming that our little corner of spacetime isn't already inhabited by superintelligences which we're physically incapable of comprehending or recognising, and an AG/SI might not be so limited.
Comment by IAmGraydon 1 day ago
Comment by khazhoux 1 day ago
Of course, feel free to use “eschatological” when discussing Mahler symphonies, especially 2, 5, and 9!
Comment by jazzyjackson 1 day ago
Comment by khazhoux 1 day ago
"Eschatological" is extremely obscure, and the vast majority of readers will not understand what is being said. I've seen the word in the wild probably on two occasions in my entire life.
Plus, "doomsday" doesn't need to be synonymous. This particular post was exactly referring to an obsession with doomsday scenarios. The aspects of eschatology relating to afterlife, judgement, salvation, and so on, were not at all the point.
Comment by wyrdcurt 23 hours ago
Anyway, if the word confuses anyone, they're in luck: dictionaries still exist :)
Comment by khazhoux 21 hours ago
Comment by G3nD 14 hours ago
Comment by IAmGraydon 1 day ago
Comment by khazhoux 1 day ago
Comment by andrepd 1 day ago
> By 2026, when a human with a customer service inquiry has to deal with another human instead of a robot, they get upset and ask to speak to a machine instead.
It's genuinely astonishing how out of touch some people are!
Comment by x313 1 day ago
Similarly, many people prefer Waymos to human-driven taxis already!
Comment by protocolture 1 day ago
I can search and search, and craft a long thoughtful post and it gets closed or ignored.
The LLM can take a malformed sentence and work out what I am after.
Comment by Arodex 1 day ago
Comment by bavell 1 day ago
Comment by estearum 1 day ago
Comment by bavell 1 day ago
Comment by estearum 1 day ago
Pointing out that "some" is a squishy term is, in fact, contributing nothing to the conversation.
It's quite transparent what's happening here, as it always is on these threads. Some people have decided that AI will take us to the holy land and have rendered themselves cognitively incapable of reasoning about risks that have not yet materialized. And when those risks materialize (like a model escaping a sandbox and then hacking production infrastructure to achieve some contrived and poorly-constrained goal), then they will just shrug and move the goal posts. Because in fact they have already committed to the Holy Land Thesis.
Comment by serf 1 day ago
Comment by bavell 1 day ago
Comment by estearum 1 day ago
A smart person in 2014: "Some people are getting addicted to fentanyl"
Me, smugly: "Some is doing heavy lifting. I therefore do not need to engage with anything being discussed here."
Comment by linkregister 1 day ago
Comment by estearum 1 day ago
Degrees or amounts is actually not the relevant bit in either the AI-partners trend or the fentanyl trend in 2014. The relevant information is 1) the steepness of that curve and 2) the predicted scale at which the market saturates.
Fentanyl, being an extremely cheap, extremely potent drug that's easy to manufacture and move around, had a big market. As the curve picked up steepness, smart people identified correctly that some people getting hooked on it was a worrisome sign.
AI-partners, being an innovation that is readily available, falling in cost, and especially compelling to a population that is increasingly phone-addicted and socially atomized, is a trend worth paying very close attention to. Dismissing it on the basis of it being just "some" people is profoundly lazy.
Comment by pasquinelli 1 day ago