George Orwell Predicted the Rise of "AI Slop" in Nineteen Eighty-Four

Posted by doener 21 hours ago

Counter82Comment59OpenOriginal

Comments

Comment by raincole 18 hours ago

I'd like to share the most prophetical quote I've read about generative AI:

> Typewriters and printing presses take away some, but your robot would deprive us of all. Your robot takes over the galleys. Soon it, or other robots, would take over the original writing, the searching of the sources, the checking and crosschecking of passages, perhaps even the deduction of conclusions. What would that leave the scholar? One thing only, the barren decisions concerning what orders to give the robot next!

-- Galley Slave, a short story by Isaac Asimov, 1942

Comment by sebmellen 18 hours ago

Incredibly to see the kind of prophecy Asimov had.

Comment by mvcosta91 17 hours ago

The Wikipedia one always get me. He was like Nostradamus for nerds.

Comment by bicepjai 14 hours ago

Inspired to come up with these original ideas

Comment by tracerbulletx 19 hours ago

“Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.” - Dune

"What do such machines really do? They increase the number of things we can do without thinking. Things we do without thinking—there's the real danger." - God Emperor of Dune

Comment by yongjik 15 hours ago

That's great, but I think Herbert had a vision of this incredible galaxy teeming with truthsayers, human computers, and space travel, and just needed a convenient excuse to explain away the total lack of computing devices.

Comment by Eddy_Viscosity2 8 hours ago

It was a convenient choice because its so obviously a foreseeable problem with thinking machines. see 'torment nexus'

Comment by mrexcess 16 hours ago

Hard times don’t create hard people, they create scarred people. I’ll take the robot farmers, undoing of wage slavery, and time to maintain participatory democracy over my favorite author’s romanticized suffering.

Comment by stvltvs 6 hours ago

We could have undone wage slavery a long time ago if automation of work was a sufficient condition.

Comment by mrexcess 4 hours ago

Here’s why I don’t think so. If we look at the milestone efficiency gains over the past century across a broad base of industries, virtually none of those could have been accomplished by contemporary automation technologies. We are only beginning to cross that threshold. It was the sacrifice of our forefathers who brought us there, just as it was the sacrifice of theirs who brought us from dank caves and death in our 30s from curable illness, into the enlightened world.

Comment by mmcromp 16 hours ago

I'll believe it when I see it

Comment by windowshopping 20 hours ago

It's amazing to me how nobody seems to know about the short story "The great automatic grammatizator" by Roald Dahl. Nobody got closer than him. I feel like I should be reading about it all the time and no one seems to have ever heard of it.

Comment by Noumenon72 15 hours ago

“There are many other little refinements too, Mr Bohlen. You’ll see them all when you study the plans carefully. For example, there’s a trick that nearly every writer uses, of inserting at least one long, obscure word into each story. This makes the reader think that the man is very wise and clever. So I have the machine do the same thing. There’ll be a whole stack of long words stored away just for this purpose.”

“Where?”

“In the ‘word-memory’ section,” he said, epexegetically.

https://gwern.net/doc/fiction/science-fiction/1953-dahl-theg...

Comment by chistev 6 hours ago

I just read this. Thanks for bringing it up.

Comment by darkerside 20 hours ago

Roald

Comment by windowshopping 19 hours ago

Autocorrect error

Comment by dlcarrier 16 hours ago

Grammatizator error

Comment by gensym 20 hours ago

[flagged]

Comment by kwanbix 19 hours ago

Why would you out of nowhere spoil a book like that?

Comment by sethjgore 18 hours ago

Because the horror of dahl’s adult stories are as pervasive even if knowing the ending. I reread many times and still get the same sense of impending doom barbarically twisting fates in the mind - what if it was true?

Comment by dbalatero 18 hours ago

Yeah sounds like a real winner of a short story at the top of the priority list.

Comment by protocolture 15 hours ago

Because its ancient and theres no social contract preventing spoilers after 8 weeks.

Comment by senectus1 19 hours ago

what was this called?

Comment by gensym 19 hours ago

The Great Switcheroo

Comment by jbaber 19 hours ago

They didn't mention my favorite part, the name. "Prolefeed" I've been waiting for someone to pick up the word so people would get more self-conscious about consuming it.

Comment by curio_Pol_curio 18 hours ago

"I read a self-help book. It's really good. Everything works out if you'd just follow the rules."

https://youtu.be/KOiDWGs4JE4?t=28s

wHNston

Comment by johnea 20 hours ago

> and a steady stream of paci­fy­ing media

Seems like he also predicted internet brain damage...

Comment by dcre 18 hours ago

He wasn’t predicting slop; he was describing mass culture, which already existed when he was writing.

Comment by refulgentis 19 hours ago

I'm old enough to feel "get off my lawn" about this: a constant for every invention is my lifetime is "everyone else is only interested in puerile sex and entertainment, $LATEST_MEDIA is ruining us, 1984" - heard this about TV, internet, iPhone in my lifetime.

It's odd to hear that applied here, it's sort of torturous to apply to LLMs. They engender sloppy creation (giving us the titular AI slop), not puerile consumption.

Comment by slipknotfan 19 hours ago

"While sloppy writing does not invariably mean sloppy thinking, we've generally found the correlation to be strong — and we have no use for sloppy thinkers."

http://www.catb.org/~esr/faqs/hacker-howto.html

Comment by mercer 12 hours ago

A lifetime is not that long though, and I'd argue that TV was the start of a chapter that internet, iphones, etc. are just ever-increasingly addictive and immediate iterations on.

I'm not saying that we didn't have anything like that before tv, or that specific individuals or groups throughout history might not have had something similar, but I do feel TV, and especially its audio-visual nature, really changed something in a way that, say, the printing press never quite did.

EDIT: and to add, my feeling on how many people seem to use LLM's is that in a way it's extra insidious because it's /tailored/, often 'puerile' interaction.

Comment by hresvelgr 19 hours ago

> a constant for every invention is my lifetime is "everyone else is only interested in puerile sex and entertainment, $LATEST_MEDIA is ruining us, 1984"

Every damaging invention in isolation isn't a big deal. The big deal is setting precedent and the accumulation.

> not puerile consumption.

I agree, it's more akin to seeing how much sawdust one can put in a rice crispy before someone notices. No one wants to eat sawdust, nor is there a mindless desire to.

Comment by black_13 19 hours ago

[dead]

Comment by irishcoffee 20 hours ago

[flagged]

Comment by dylan604 20 hours ago

Animal Farm was required reading at my school. They also did Fahrenheit 451 instead of 1984 though.

Comment by operatingthetan 20 hours ago

The four books you mentioned have very different methods of control. The primary thing they share is being dystopic.

---

1984: control through fear and pain.

Brave New World: control through pleasure and distraction.

Animal Farm: control through corruption and deception.

Atlas Shrugged: control though guilt and regulation.

---

Brave New World is the most prophetic.

Atlas Shrugged has horrific writing, separate from what I feel about the politics.

Comment by Terr_ 20 hours ago

> Atlas Shrugged has horrific writing, separate from what I feel about the politics.

Following the tangent: I read the book "blind", when I was mind-numbling bored for a couple pre-dialup weeks at a relative's house. Eventually I decided to finish it purely out of spite so that I could confidently denounce it as trash in the future. (And today it pays off?)

In short, it's a book of incredible hypocrisy which also disrespects the reader's intelligence and time.

Hypocrisy, because Rand asserts that certain appeals to emotion or outcome are evil tools of fictional villains, while simultaneously doing the exact same thing in the real world to the audience. The difference is that instead of "think of the starving children", it's "think of the Marty Stu [0] corporate executive üermenschen", the characters the author has been playing up for a couple hundred pages already.

This is compounded by the manifesto chapter where Marty Stu does nothing but monologue. The jarring transition reveals how the story was really just an afterthought, a kind of necessary deceit to get people ready to swallow a pompous diatribe without looking at it too closely.

[0] https://en.wikipedia.org/wiki/Mary_Sue

Comment by piekvorst 16 hours ago

> certain appeals to emotion or outcome are evil tools of fictional villains, while simultaneously doing the exact same thing

You would be correct if that were the whole truth about Atlas Shrugged: defending protagonists on emotional grounds.

But it’s not the whole truth. The very monologue that you dismiss is the tool that provides the emotion with the principle. You know the characters’ reasons for holding their emotions.

Ayn Rand never said that one shouldn’t feel or express one’s emotions. On the contrary, “. . . emotions are not his enemies, they are his means of enjoying life” [1].

In fact, every emotional appeal used in the novel is supported by argument, sooner or later. You cannot say, for example, that the dismissal of James Taggart or Robert Stadler is purely emotional.

> The jarring transition reveals how the story was really just an afterthought

Your claim would be valid if the jarring transition were not Galt’s speech but some other nonfiction. The case is the opposite: the story and speech are very much integral.

The pause of events as such is a neutral tool, with precedents (The Battle of Waterloo in Les Miserables).

[1]: https://aynrandlexicon.com/lexicon/emotions.html

Comment by nemomarx 20 hours ago

1984 has fear and pain for the white collar set like the protagonist, but it's implied mass media, telescreens, and propaganda do for the working class there, which is similar to BNW's style and of course has overlap with Animal Farm.

A pretty good study of different flavors when taken together, though?

Comment by Rury 19 hours ago

Honestly though, they're not all that prophetic. I mean, you can find widespread instances of each means used throughout human history. Although I would happen to agree that the methods that feed complacency and ignorance are the most effective.

Comment by slipknotfan 19 hours ago

You have to defend your freedom from all angles of attack.

Comment by mrexcess 16 hours ago

Wasn’t 1984 a bit more about control through surveillance and silencing, than about pain? Everything was a lie, and every refusal to accept the lie was a signal to Big Brother.

Cast as such it seems rather more prophetic than Soma, IMO.

Comment by irishcoffee 4 hours ago

BNW wasn't about pain, it was almost completely about control.

Comment by irishcoffee 20 hours ago

That was kind of the theme of the suggestions, control. I’m kind of stoked you identified it.

Comment by vincent-manis 19 hours ago

“There are two novels that can change a bookish fourteen-year old’s life: The Lord of the Rings and Atlas Shrugged. One is a childish fantasy that often engenders a lifelong obsession with its unbelievable heroes, leading to an emotionally stunted, socially crippled adulthood, unable to deal with the real world. The other, of course, involves orcs." -- John Rogers

Comment by gottorf 18 hours ago

It's a fun quote. On the other hand, while no one should accuse Rand of being a good writer of fiction, I don't find what's depicted in Atlas Shrugged all that fantastical.

Comment by piekvorst 17 hours ago

This quote would be more meaningful if Atlas Shrugged critics were able to actually criticize it, not the straw-man. Unfortunately, orcs didn’t show them how to do it.

Comment by irishcoffee 19 hours ago

For what it’s worth I read Lotr when I was 8 and atlas shrugged when I was 12. I’m must be stupid naive about the discourse over shrugged around here. The meta-story made sense to me as much as the hero journey of frodo(and gollum) made sense to me.

I mean this sincerely, I don’t understand the beef with shrugged. The idea of “a small population owns the world” not only made sense as a theme, but it what is happening in the world today. I must be too stupid to have realized the political bits.

Comment by crooked-v 19 hours ago

The beef is mainly that the book portrays Galt's Gulch as both a good thing and as something that would actually function, when the real world consequences of trying to run a society like that are that your town fills up with wild bears that destroy everything and eat your pets (https://newrepublic.com/article/159662/libertarian-walks-int...).

Comment by piekvorst 16 hours ago

It is a good thing and it absolutely can function, but not as a society. Ayn Rand explicitly rejected this idea [1]:

    Q: Why is the lack of government in Galt’s Gulch (in Atlas Shrugged) any different from anarchy, which you object to?

    A: Galt’s Gulch is not a society; it’s a private estate. It’s owned by one man who carefully selected the people admitted. Even then, they had a judge as an arbitrator, if anything came up; only nothing came up among them, because they shared the same philosophy. . . . But project a society of millions, in which there is every kind of viewpoint, every kind of brain, every kind of morality—and no government. . . . No one can guard rights, except a government under objective laws. . . . Rational men are not afraid of government. In a proper society, a rational man doesn’t have to know the government exists, because the laws are clear and he never breaks them. [FHF 72]
[1]: Ayn Rand Answers: The Best of Her Q&A, Politics and Economics, Libertarianism and Anarchism

Comment by crooked-v 15 hours ago

That's a real circular answer there, "it's not a society because it's not a society". Claiming that a place specifically set up as a self-contained community hiding from the government, and hiding from government authority, is 'just a private estate' is rhetorical nonsense.

Comment by piekvorst 15 hours ago

I never said that it’s not society because it’s not a society. If you want me to derive the whole meaning from concepts, that’s a rationalist school’s method, not mine. I derive the meaning from the content: an example of (“optimistically”) one thousand people sharing the same philosophy does not endorse a government-less society.

I do agree that a private estate cannot exist in a society without government. But it is private estate in the sense that the members recognize it as such, and that it can exist in this very narrow context.

Comment by refulgentis 19 hours ago

I liked Atlas Shrugged, didn't go over well for me because I'd read all of them by 15, and I assumed 2/4 were de rigeur in at most high school.

Comment by thrance 20 hours ago

Brave New World warns against the dangers of consumerism, hedonism and complacency.

1984 warns against fascist modes of governance, the dehumanization of individuals under totalitarian regimes.

Animal Farm warns against the danger of revolutionism, and the way ideals can be led astray.

Atlas Shrugged warns against... The way poor people steal from the rich? How rich people are the only productive members of society? How we'd be better off if we just ceded total control of our society to the oligarchy?

Yeah... One of these doesn't belong on the list. I read all four, and while I enjoyed the first three, the last one is closer to fanfiction than literature in my mind. I always think of AnCap memes and chuckle to myself when I see it mentioned.

Comment by tracerbulletx 19 hours ago

The villains in Atlas Shrugged are other rich people who achieved their power with corruption and mysticism. I do not want to enact the morality of Atlas Shrugged, but its also wildly misunderstood by almost everyone for some reason. Its mostly just supposed to be competency porn.

Comment by thrance 12 hours ago

Hmm, the book in its whole is basically a manifesto for Rand's Objectivism. I think I'm mostly correct in painting it as a "rich people good, poor people bad". As I understand it, the villains in her book are the hordes of "looters" (aka proletariat) that is too lazy to create their own wealth and instead just steals from our cool class of Übermensches, that is solely responsible for the entire world's productivity, through the government. I find Rand's worldview a bit silly, but it's an understandable reaction to where she came from.

Comment by tracerbulletx 6 hours ago

The main antagonists are James Rearden who is running the railroad and Lillian Rearden the rich wife and a socialite. Then high ranking professors, lobbyists, and generally the military and religious institutions. She thinks that rich people who get rich by scheming and lying are the villains. The titular character is working as a poor railroad hand for most of the book and that is seen as heroic.

Comment by mrexcess 16 hours ago

Atlas Shrugged, more charitably perhaps despite its manifest flaws, seemed to me to be about the dangers of putting “the needs of the many” over individual rights, and how it can ultimately be self defeating for the whole.

Comment by thrance 12 hours ago

Ehhh, is this really what the world is suffering from though? Too much equality?

Comment by piekvorst 5 hours ago

Too much social conflicts caused by uncritical pursuit of “equality” (really, privileges).

Comment by mrexcess 4 hours ago

I don’t think, “too much equality” was one of the themes. Rather, it was about too much centralized power over the individual. And yes I do think that’s somewhat relevant to understanding issues of today, including mass surveillance the centralization of technological control behind crypto-nationalized zaibatsus, etc.

Comment by 20 hours ago

Comment by thrance 20 hours ago

Fitting how the author felt compelled to use Gemini to generate an ugly banner for their blog post. An image completely devoid of meaning, that adds nothing to the article except a few kilobytes: slop under any definition.