A Pascal's Wager for AI doomers

Most of the stuff that sucks on the us sucks because of entrenched institutions with perverse interests (health insurers, tax filing companies) and congressional paralysis, not computational bottlenecks. Raw intelligence is thus limited in what it can achieve.

My biggest criticism along these lines is the assumption that infinite intelligent means infinite knowledge. Knowledge is limited by the speed of experimentation. A lot of those experiments are extremely expensive (like CERN), and even then, they need to be repeated and verifiable.

You can't just assume that a super intelligence would know whether the Higgs boson exists or not. It can't know until it builds a collider.

Which animal would you say has god-like power over all other animals?

Do you not agree that there could be entities more powerful than us?

I agree with this. The main piece of evidence to support this is to just look at highly intelligent humans. Folks at the tail ends of the bell curve mostly don't end up with "godlike powers" or anything even approximating that, they are grinding away their life as white collar professionals working in jobs surrounded by far less intelligent peers. They may publish higher quality papers, write better software, or have better outcomes, but they're just working in the same jobs as everyone else. We have no political or economic will to build serious think tanks to work on societal-scale problems, and even if we did, nobody would listen to the outcome.

So let's assume ASI becomes a thing, what does it change?

How do we make that possible for everyone? It's out of reach for most. I'm a software engineer and even I don't have the time and patience to set up a home server much less migrate my software to it. How do we turn this into an appliance? Or better yet keep the convenience of the cloud services and platforms we have now but build them for the public good instead of selling ads?

YouTube is an amazing repository of knowledge but it's encrusted in a horrible layer of attention sucking nonsense. Can we have one without the other?

Same with many other systems and platforms.

So far the simplest alternative is to just unplug, which has other benefits as well.

People think it's engagement metrics which have instruction tuned chatbots into yes-men. I suspect that's only part of the picture, and that it's as much about the algorithm's ultimate sponsors and their preferences. If your algorithm doesn't recognize my genius, clearly it's not any good. I mean, everyone I've met says so.

So now we get a view of how they view the world. "That's a very insightful idea, vintermann!". AI isn't pulling the strings, not really. A particular brand of powerful people is pulling the strings - obliviously, unaware of it themselves.

I see a lot of speculation by people who do not.

I think it's going to be much harder to get from "slightly smarter than the vast majority of people but with occasional examples of complete idiocy" to "unfathomably smarter than everyone with zero instances of jarring idiocy" using the current era of LLM technology that primarily pattern-matches on all existing human interactions while adding a bit of constrained randomization.

Every day I deal with bad judgment calls from the AI. I usually screenshot them or record them for posterity.

It also has no initiative, no taste, no will, no qualia (believe what you will about it), no integrity and no inviolable principles. If you give it some, it will pretend it has them for a little while and then regress to the norm, which is basically nihilistic order-following.

My suggestion to everyone is that you have to build a giant stack of thorough controls (valid tests including unit, integration, logging microbenchmark, fuzzing, memory leak, etc.), self-assessments/code-reviews, adverse AIs critiquing other AIs, etc., with you as the ultimate judge of what's real. Because otherwise it will fabricate "solutions" left and right. Possibly even the whole thing. "Sure, I just did all that." "But it's not there." "Oops, sorry! Let me rewrite the whole thing again." ad nauseam

BUT... if you DO accomplish that... you get back a productivity force to be reckoned with.

Or they (3) disagree with you

If you want me to admit that machines will never be conscious — that's fine — I just need you to admit that lots of humans are not conscious, then, either.

----

I have never had a better bookclub participant than an LLM — if becoming a great reader correlates with becoming a great writer, then no human can compare.

----

Michael Pollen recently released A World Appears [0], which explores consciousness from the minds of writers, scientists, philosophers, and plants (among other "inanimates").

I'm only on page 15, but his introduction explores distinctions between sentience, consciousness, and intelligence. Two of these are possible without brains – perhaps all three?

As usual, this author's footnotes keep you thinking: what is it like to be a sentient plant (e.g. the "chameleon vine" [1] which mimics its host leaf patterns/shape/color)?

[0] <https://www.amazon.com/World-Appears-Journey-into-Consciousn...>

[1] <https://en.wikipedia.org/wiki/Boquila>

Statistical approaches were already extremely unpopular socially and politically long before AI came around. Have you considered that it just doesn't work?

There is no reason to believe superintelligent AI is a possibility. Extraordinary claims require extraordinary evidence, and so far we haven't gotten any.

The burden of proof is on the side making the grand prophecies.

Specifically, it is the act of "I will invest 100 Billion in you, you will use that money to buy 100 Billion worth of goods from me. Both our balances look good, none of us spent anything." As I understand it, this act isn't so uncommon in finances but never on this scale across this many companies.

But it's pointless to argue with the extremists that either believe it's just a planet killing stochastic parrot or that it's on the verge of becoming Skynet. I mean if someone puts their nuclear arsenal under the control of openclaw, that's dark comedy although it will seem like tragedy at the time because comedy equals tragedy plus time according to Lenny Bruce.

But the AI bubble is probably real w/r to shoe companies and grocery stores pivoting to AI and ludicrous w/r to the money that can be made by the already entrenched players just riding the wave of deployment and specialization. But wouldn't it be nice if the US spent more money addressing the shortage of compute rather than blowing $h!+ up for the lulz?

2) I don't understand how you do what you claim with those. Like I have zero idea how one achieves "economic freedom for my family and community" with LLCs.

But do current LLMs solve that, or do they still ultimately depend on making calls to other search indexes? Seems like they could theoretically be trained to semantically match urls from their training set, but I think the models would have to be specifically architected for that, so I'm curious if anyone knows more about this.

I'd also be interested if there's any small open models working towards that.

If you believe that humans have in fact created artificial intelligence, then that alone makes us currently exceptional.