They’re made out of weights

> Blind people learn concepts of bikes too and in a smiliar way: by description.

And by touch and sound. And maybe some were daring enough to drive one, or unlucky enough to get hit by one. But have way more input than just texts.

Compression and existence of mechanism to expound on it does not imply consciousness.

Otherwise, yes, finally people observe the very apparent fact that LLMs are one very smart compression.

It is basically reductionism, or in the extreme atomism, and (as I understand it) it has largely fallen out of fashion both in the philosophy of science and in the philosophy of mind, since it heyday in the 1970s. And I don’t see it coming back into fashion any time soon.

I can definitely see why reductionism would appeal to an educated public. We learn about the periodic table; we are able to break sentences down to words, down to letters; to break an executable down to binary code, to machine instruction, to electrical currents flowing through semiconductors. Why shouldn’t we be able to do the same with conscious thoughts? It is certainly an appealing thought process.

As I understand it, reductionism started to fall out of favor because of the rise of quantum mechanics and chaos theory, where we have a lot of weird phenomena which cannot be explained by reducing the particles down to the sub-sub atomic (or rather they are better explained by describing the interactions directly).

Saying we have a good idea of how the brain works massively overstates the case...

We know how neurons fire. We do not know how a brain turns that into thought, meaning, intention, experience and on and on. That is not "pretty well understanding the brain", it's understanding some components and hand waving the thing we actually care about.

> The purpose of [mathematical] models that are built thoughtfully is to explain why complex systems are the way they are, with data and algorithms, however imperfectly.

Nope. The main purpose of the whole endeavor is usually to predict the behavior of a complex system, because that's actually what we care about. If we can predict it, we can adapt to it, and eventually use it to our advantage.

Explaining why a complex system is the way it is, is merely nice-to-have. Models are opinions. All of them are wrong, but some are useful, and we rank them by how useful they are. The models and explanations are important because, beyond their elegance and convenience, it's also the case that more accurate models give you better predictions across larger domains, meaning we get better at getting something useful out of the complex system.

People get fixated on modern theoretical science, with bottom-up mathematical explanations traced through seas of empirical data, with whole magical rituals of peer review and double-blind studies and statistical significance around them. But they forget that the core of empirical science is literally throwing shit at a wall to see what sticks. That is the guiding principle, everything else is just making the process more efficient.

Understanding complex natural systems (or even engineered ones that got too complex) always starts with tests - tests on the real thing, then on approximate models that we poke and prod and bash into shape until they start acting similarly to the real thing. It's through the poking and bashing, and how they affect our proxy model, that we glean insights into nature of the simulated phenomena, and eventually formulate general theories - but more importantly, the models give us useful predictions from the start, before we have any theories explaining why.

> The purpose of LLM models is to give the illusion of answering questions while never answering why the answer was given.

This is just your own idiosyncratic and biased belief. You're not describing anything objective about LLMs, you're describing your personal attitude to them. This colors your understanding in a way that can't really be reasoned with until you let go of the artificial constraints you're imposing on your own understanding.

> Sit next to a bank of slot machines for an hour and listen to the absolutely ridiculous shit most people will come up with to explain how they "know" if a machine is going to pay out soon, and then tell me if you think it's a good idea to give them an LLM in their pocket to answer their questions in whatever way they frame them.

If the LLM in their pocket has a more robust world model than they themselves and is e.g. able to refute their irrational convictions, it actually seems like a very good idea. (Big if, of course.)

We already live in a world of abundance.

No one has to work on a field for their food.

Its a capitalism problem we have, not a resource one.

Have you heard of property tax? We have that here.

I also do not earn currently a farm, im planing to buy one, i'm still 100k short though.

Its not me creating the misery, its our capitalistic system.

Because capitalism doesn't allow for that.

Only a fundamental change to our society will allow this for the masses when pressure to the rich skyerockets

right, these knowledge work and coding jobs are, by my lights, about the best possible job. From my perspective we've invented a machine that does the fun parts while leaving me the less fun parts (review, various hard-to-claude janitorial tasks, etc).

I might like woodworking as a hobby (for example), but I sure as heck don't want to be a carpenter or to depend on my ability to hand craft enough widgets people like to survive

I differenciate between things you have to do (work) and things you want to do. Work means someone else is telling you your priorities.

If you want to write code and think, you would be welcome in my utopian vision.

But when i write code, its business shit. And its business shit someoneelse already solved a few times.

One of the best thing I done for my career (as a self taught software developer, but with a degree in electronic engineering) is to learn computation theory.

Computation is math (and a very restricted subset of math). It’s mostly specific sequences of sets manipulation. What sets and what manipulations are defined by people, not by the idea of computation.

The best thing is that as soon as you specify the sequences of manipulation, it become a a set that you can manipulate. That can be a difficult concept to grasp, but that’s what helps in designing notation that are more appropriate for the human mind to describe a solution for a specific problem.

> Reality has no distinction between "data" and "code"

yes, yes, ostensibly the universe is built on lisp.

But we all know that it was hacked together with a lot of perl[1].

[1] you all know the reference.

Fermat's principle is an outcome of constructive interference of waves. It works both for classical and quantummechanical descriptions. E.g. check https://phys.libretexts.org/Bookshelves/University_Physics/U...

> a ray of light has to know where it will ultimately end up before it can choose the direction to begin moving in

I'm no physicists, so I guess I'll ask it: Why?

Also related, why do some ray of light then "see" a black hole yet decide to head into them anyways, if they saw it before they went in that direction? Seems like a dumb move :)

I read your comment as criticizing the OP's story as pointless and unoriginal. My comment elucidated the point of the original story, and what I think is the point of the second story.

That's a very interesting reading! It's been a while since I read it, but IIRC Watts explicitly talks about the matter of intelligence vs consciousness at length and Rorschach's lack of consciousness is essentially proven at some point by the linguist. The distinction is driven home subjectively, personally by the viewpoint character. Your reading adds a rather diabolical twist! I don't think it works though since Rorschach was clearly a tiny part of a much larger, more powerful entity and the humans are clearly doomed. I'm not sure it serves a narrative purpose to have humans be a weak, evil civilization destroyed as a side-effect of a good, strong civilization's actions. I read it as a (hopeless) conflict between non-conscious, infinitely strong aliens vs conscious weak humans and post-humans. I mean it took extreme, heroic effort to damage what is probably a tiny appendage of a much larger civilization/organism, and you lose the emotional resonance if humans are the baddies.

You underestimate the intelligence of your cat. Or rather you measure intelligence with an extreme human bias. What you consider intelligent behavior your cat may consider weird, and what your cat considers intelligent behavior is something you will never consider.

That said, I don’t think it is useful for philosophy nor science to consider intelligence to be the same thing as consciousness. In fact I would go even further and claim that intelligence is not a useful construct, neither for philosophy nor for science. Consciousness, on the other hand, I think is useful for philosophy, but not (as of now) for science.

your link has a si= tracking parameter

Just wanted to say, I appreciate your patience and good sense in this thread.

It's difficult to tell who's trolling -- probably best to go with the charitable assumption that everyone is honestly trying to convey their opinion, but mostly talking past each other. Unfortunately these discussions about the nature of consciousness never go anywhere useful.

I think I'm probably in the same boat as you, roughly: a) LLMs are doing something really interesting that resembles in many ways both intelligence and consciousness; b) I suspect they're not actually conscious but I don't know how you'd know for sure; c) it all just drives home that we still don't really know what consciousness actually is. But like (a), it's definitely something really interesting...

I read some stuff once where the author argued that almost everything takes in values, performs operation on them and gives output. Some things also store things. Then, if everything is doing logic operations there is no telling where intelligent "life" might be hiding. It also is really one giant system.

The toaster has hard coded or configurable weights. It makes a product from heat and time.

You could make toast by heating a brick on a campfire. It would be a clear sign of intelligence.

If we lift one weight out of your brain we wouldn't look at the number and say it is intelligence. It must exist in a chain or matrix multiplication to qualify.

The fun thing is that the timer and feedback mechanism do exist in a chain of events.

It's part of the system and it takes all steps to complete the task.

Our chain of thought that leads to making toast would be abandoned if it wasn't in working order.

It would be equally compelling, because the compelling nature of the story comes from the language, the presentation, rather than the [specific thing being ascribed consciousness].

An LLM is as conscious as a toaster...

> But it would fall flat because the toaster is not a compelling subject in a discussion of consciousness.

Teapots are not compelling.

> You don't have to believe that LLMs or AI agents are conscious to acknowledge that the argument for their consciousness is far more compelling than any other technological artifact.

God is compelling t billions of people.

Is Russel’s Teapot a bad argument in the God debate?

    No one cares about teapots in space either (Russel).

    Teapots are not compelling.

Sorry, you're right. Thank you for pointing that. I'm unfortunately past the edit window.

Oh, no. When you add thermal sensing and offsetting ambient temperature, you can add all kind of "seemingly" intelligent features like feeling the emotions of the bread, and creating the perfect toast without hurting the cute little bread slices, making them perfectly blonde while not showing cruelty to them.

They can even adapt to their environment and the characteristics of the bread even with simplest of mechanisms because the text will be overglossing the fact that different types of breads have different thermal characteristics and this will deeply affect the behavior of the metal strips, bordering near a sentient being even more thoughtful and considerate than a human which is rushing through house to catch the bus in the morning.

The key point here, I think, and why it's necessary to have read the original story, is that being able to express an idea is not the same as that idea being correct.

You could go through and change all the points in the original story to be about a toaster instead. It would require you to edit text, but there is no barrier to doing that.

What I think a lot of people are getting stuck on is that editing the text to say 'toaster' would not mean that toasters were conscious, and that editing the text to say 'weights' doesn't mean that weights are conscious either. Stories aren't factual just because they are written.

The original story was written by the thing claiming to be conscious; the LLM and toaster ones would not be, which undermines the claim to consciousness a lot.

[dead]

Ah thanks. That context was missing for me appreciated.

I appreciate you taking a moment to write this. I was a little confused by the downvote. I think I have a tendency to credit satire at times when it's not intended... my own sense of humor has a lot to do with tweaking people's expectations, and coming from a family of tricksters, no one wants to be the one who doesn't get the joke. So maybe it's a me problem. Having said that, the situation with the aliens is that they can't conceive of intelligent meat, because they can't conceive of how that could work. We do understand how matrix multiplication works and how it gives rise to apparently emergent behavior, because we theorized it and we engineered it. So I can't help taking the idea that we'd be baffled at "that's it, just numbers?" as anything but tongue in cheek.

I'd only add that if it's not intentional satire, it's an even more profound example of the unintentional variety.

I wouldn't use the word necessary.

IMO, we are probably talking about a 6x slow down (for typical english). You would need to be absolutely stupid not to implement some kind of optimisation along these lines.

Slower and maybe a little dumber; But it would work.

We should tell the Unix people that they've been giving /usr/share/dict the wrong name for over three decades. (-:

The point that when the rules the model learns are simple enough they stop being spread out over all the layers and become as easily interpretable as any expert system.

It's just that the rules we feed in the model are extremely poorly defined and we end up with the soup of disjoint rules smeared all across the weights.

This isn't a feature of the models. It's a feature of the training set.

Being shocked that you can store rules in floating point numbers is the same as being shocked you can store rules in integers. It's been a century since Goedel Numbering was invented, we should be used to it by now.

>Integers in normal programming represent data or instructions; instructions are hand coded, have rigidly defined semantics, are not differentiable and have no redundancy.

I can, and have, written programs using an evolutionary algorithm that then run on bare metal. None of the things you list are true for those programs, yet other than being computationally more expensive to train they work just as well as neural networks.

>I don't understand what you mean to say

The diffusness of weights across the whole model isn't an innate feature of deep learning models. It is a feature of sparse training data and little compute.

I don't see how. You could ask a neural network to do the tokenization I suppose, but in doing so you'd have to convert the prompt into tokens via the same deterministic process the network was trained on, essentially just moving the exact same process up one layer.

Internal experience affects movement of physical molecules in what way?

Internal as in that which is feeling things. In this context the physical brain is what is responsible for what you do and feel, but it is not physically different from other matter and it requires no "internal observer" do do its thing.

I'm not saying that they don't have some internal self model that helps them model the internal states of others, quite the opposite. I am saying this specific explanation seems to lean on a biological mechanism that (at least by the phrasing of the linked article) is only present in "hominids".

I don’t think thinking is consciousness, but I don’t see how consciousness operates without thinking.

Consciousness might feel like it’s an isolatable experience, but everything we experience is a result of processing it. Pause a brain and it isn’t experiencing anything,

We can be conscious without overt reasoning, but not without thinking. Thinking about ourselves and the experience is a basic component of the self-awareness in consciousness.

> Consciousness doesn’t require thinking, it only requires perception.

Could we imagine a human being that is conscious but has no sensory input nor any occurring thoughts but is still conscious?

Only if you ignore almost every input and output that neurons have.

https://www.quantamagazine.org/ai-is-nothing-like-a-brain-an... https://pmc.ncbi.nlm.nih.gov/articles/PMC9665914/

This is why making more neuromorphic NNs is still an active area of research, although they typically all focus on another extremely simplified model (spiking neural networks).

Agree and add, don't confuse the substrate for the computation. Of course it's also clear that we don't quite have a full and definite picture of what the computation consists of in the case of a biological brain as evidenced by our continued failure to accurately simulate even the simplest of organisms.

I'm not sure consciousness and self-awareness are the same thing. First is we can be conscious when we sleep/during REM sleep, where it's arguable we are not self-aware. And if not that, we can even do it when awake, for example when we think about a movie, or a philosophical problem, we can have conscious thoughts that are not related to the self. This leads me to believe consciousness is separate from self-awareness. Self-awareness is _one thing_, among many, that the brain can think about and be conscious of.

And also "instilled during their reinforcement training", and we are currently pushing planning hard there, for autonomous agents.

When we say "it rains", do we consider that "it" has any intention of agentivity?

Some questions are just ill formed.

Plus even if "LLMs are alive and conscious", this still would scratch the surface of the morale/ethical/societal considerations that people really care about.

Because even with other humans, we can argue if they exist or if they are mere npc in a solipsist world view.

I think consciousness is going to turn out to be very challenging to define rigorously enough that we can test for its presence or absence. Emergent or not, the question is how do you determine when it has emerged? Is it a quantity or an attribute? Discrete or continuous? Does it have a finite or infinite range?

We can all agree on what color something is, but we can’t describe the color a priori, only by example. I think consciousness may be a similar phenomenon and the only test is by shared experience. If so then we are in deep trouble because we will not be able to anticipate when a system becomes conscious.

I think the alternative is that our brain, somehow, is connected to some metaphysical aspect of reality which is what most religions believe.

I don't know if most people would dismiss the sentence "all matter has a sort of proto-subjectivity very different from ours but which gives rise to ours". And it solves some problems (introducing others).

I'd expand a little bit to say inevitably emergent property. That is to say, if you create a sufficiently complex information-processing network, some level of consciousness will result. With regards to current AI, we're a fair way away from building something with enough connections, but we'll get there.

One thing that gives me pause about the inevitability hypothesis is that the type of connection, or manner of information processing, may matter: there might be something about neurons that isn't (currently) reproducible in silicon. I don't know, and there's not (yet) any evidence for or against, but it at least seems like plausible speculation. We just don't know enough about any of this right now.

Well, I think the "brain as an antenna" theory is also plausible given your preconditions.

But I think my issue with the emergence theory is that it seems to imply to me that consciousness is non-physical and non-local. So what entity is actually experiencing the consciousness? It's not that I believe consciousness is physical and local, but people who make the emergence argument seem to believe it is and I can't figure out how that is supposed to work.

Yeah, I guess what I'm trying to ask is that if it is an emergent property but not a physical part of the brain, doesn't that imply something metaphysical about consciousness? Almost as if it's a non-physical phenomenon? At least when I hear people talk about emergent behaviour I see it as a refutation of the spiritual, but to me it seems like it actually implies we have a "soul".

Idk, it's really hard to articulate my thoughts here and yes it is pretty close to the conversations I had in college on various substances. Lol.

We have general purpose hardware and we have hardware that's hard wired for specific purposes like ASICs and we have everything in between.

And we are only doing it for a few decades. Evolution had million of years of "try and error".

Yes

I'd push back slightly — not on the conclusion, but on the reasoning. There's a simpler explanation that accounts for the same observations.

The matrix multilication underneath a large language model is the hardware but the data and the forming weights is not.

A quick sort sorts a list. A LLM depends on its learning data.

You train a model and then you use the model.

> Why do you need a specific organization of molecules for a phenomenon similar to consciousness to arise?

They would have to be similar because the argument is that they are similar; "AI is like the human brain" only holds true if it actually is like the human brain, not merely a superficial resemblance.

What I'm describing in that prior comment is how a lot of people drastically simplify the resemblance in order to make it feel true; That the lack of a Jesus Christ coming down from the heavens to tell everyone they have immaterial souls that the computers don't makes the comparison more true.

> We don’t really know or understand the relationship between the raw physics and again what we consider consciousness, so it’s simply a statement of “we can’t refute that these systems exhibit something similar” because we don’t know enough to refute that

Therein lies the conflict: "You can't prove it's not conscious" is an unfalsifiable statement. You can't engage with the argument because it's proponents will always claim victory, often with their own interests at play. All concerns about "superintelligence" or the long term ethics of "when do our robots become sufficiently intelligent that they'd be slaves?" have been subsumed into the AI marketing machine. However sincere one might try to address the issue, they look like a Sam Altman stooge by association.

It's like claiming the quantum fluctuations inside a pet rock as a "consciousness", even if observed directly any measurement of random noise can still be dismissed as "nuh-uh it just takes billions of years to have a thought".

More practically with current AI systems, we can look inside them pretty well and there genuinely is nothing there. Standalone LLMs are purely feed-forward systems. Their failure modes show that they perform no meaningful thought or world modelling during inference. They're just language models.

The reasoning and agentic systems are even easier to introspect. We know how they work, we can look at the full prompts & context they operate on. There is nothing there.

This is what sets AI apart from animals, which are given the benefit of the doubt on their intelligence.

> “AI/LLMs are just like the brain” is a strawman, why does this claim need to be true for LLMs or any artificial system to be considered to have something akin to the thing you think of as consciousness?

It doesn't need to be true but a lot of people make it/assume it.

There's a lot of, perhaps casual and uninformed, conversations that strongly imply a deeper understanding of the "physics" of brain chemistry than we actually have, mostly by comparing it to machines we've constructed.

(I believe) We don't need to replicate human neurons and dendrites and whatever else is in there in order to create a sapient "machine", but whether or not we've actually done that isn't being helped by arguing that what we currently have is all that similar to a human brain.

I think the point of the commentator above is that there are two extreme narratives that start each start with an uncontroversial assumption and then taking it to a pretty wild place. One narrative takes the assumption that brains are just matter so it should be possible to engineer consciousness and then argues that LLMs are conscious. The other takes the assumption that LLMs aren't conscious but then argues that because they aren't we won't ever be able to make anything conscious.

I don't actually think the commentator you responded to is arguing for either of these narratives and I thought it was a pretty useful way to look at some of these arguments.

I'm partial to "modern ML weights are much closer to 1:1 capacity mapping to synapse count than to neuron count". A single biological neuron is closer to 100 or even 1000 weights worth of ANN than to 1 weight worth of ANN.

In which case: modern LLMs are still running in a capacity-starved regime!

Even Mythos 5, the 10-trillion monster LLM, the scaling law boogeyman, the harbinger of Vera Rubin NVL72, doesn't quite rise to 100T-to-1000T of synapses. Anything the light of today's AI touches still lives in the shadow of what evolution has managed to cram into a single human skull.

We're arguing about the limitations of AI while our best AIs are still very subhuman in the scale dimension. The one dimension we know how to push. And it's already this tight.

> 1) This definition could actually be expanded (for example, with definitions from Mumford or Reuleaux). But still this definition cannot be applied directly to living organisms.

I'm not sure I understand this. Why not?

This proves too much. As it would imply parrots have sapiencee.

I think you're right in that it has the shape but I think it's missing a pretty key piece. We still haven't been able to solve catastrophic forgetting, yet everything with a brain has. Basically LLMs seem good at approximating intelligence on a moment-to-moment basis, but feel quite far away when you chat with one over time.

Like at some level, yes, transformers are trying to emulate a human brain but the second you ask folks if they do a good job of it, I think most rationale people would say no.

> something like the universal approximation theory comes to mind, transformer architecture clearly has the shape of a universal algorithm approximator

That says nothing about emulating a human brain.

If we go along with the though that our own subjective experience is the only thing we truly know, and that we cannot really know if any other humans are having the same experience (and any belief of that sorts is purely extrapolation), then there isn't a fundamental difference between LLMs and "other humans" in terms of whether or not they're "conscious". Sure, it appears more likely that "other humans" are real conscious beings, but there's no fundamental difference.

>whether these systems do or could in the future exhibit something similar?

I think the whole discussion is based on the idea that consciousness isn't something you can "exhibit". (Tell me, how can you "exhibit consciousness"?)

Diaspora is mind-blowing, yet highly improbable and speculative, even though carefully threaded to sound plausible on all levels. The whole introdus idea and simulation from VM perspective sounds incredible, but I don't think the body runs a simulation of anything. It is something else.

The issue of consciousness appears when you think of the world in a mechanistic way: since all there is are laws of physics and materiality, then how could we explain our though processes and our perceptual experience? If the world itself (in a general, existential way) is only made of laws of physics and matter, the consciousness needs to be an emergent characteristic of physical systems, and needs to follow the laws of physics. Now at this stage, you are already in trouble and you need to explain what consciousness is and how it manifestates. And that's the moment where things like the computational theory of mind appears.

But you need to step back in order to detect the fallacy, one of which is: the brain/mind processes information like a computer, then we could build better computers that can think. This fallacy is assumed in the question 'can a machine think?'. There's another fallacy, which the author call the first step fallacy, which is common nowadays: we solved the language problem, then machines will be able to think in the near future.

So it is not about solving the consciousness problem, it is about not claiming things based on assumptions that can be easily challenged.

Yes again, ribosomes have nothing in common with machines, that are built and designed by humans.

The ball is in your camp to provide solid reasons to believe why they should be grouped together, when one is a deeply complex interrelated dynamic system (in fact, arguably the most complex system we know of) evolved bottom up over billions of years that we only very partially understand and cannot fully explain or document, and the other something entirely planned, designed, and produced by humans in which every component is finite and accounted for.

The argument boils down to “well the vibes kind of match to my taste, and it’s the best analogy I have in my analogy toolkit”, which is just not serious reasoning.

There is no need to assume the supernatural. They are machines based on what? By definition, a machine is something build by humans, or any intelligent life form (because we extend the definition from our own experience as self defined 'intelligent life forms'). Living organisms and machines do share traits under our current frame of reference in modern science. But that doesn't mean cells are machines. The right framing would be 'cells exhibit behavior and certain internal relations that we can conceptualize them as machines', which is different from 'cells are machines'.

We are talking about physical replication, please let me know when a computer worm can turn my laptop into 2 laptops

Otherwise you’re just arguing that Sims are totally alive because Sims can make baby Sims.

Nothing. It is just that they are so incredibly complex, that after +100 years of research we still don't know how they work or why. Will there be a time when we finally understand them 100% and could replicate them? Could be. That does not make them machines in any case, and the fallacy of thinking that mind/consciousness can emerge from machines will still be there.

> Horizon can only be expanded at the cutting edges.

Ironically this was advice Ralph Waldo Emerson gave in 1840 so by your logic it's irrelevant just because it's old.

I have that book at home and I'll check as soon as I get back from a trip.

When Dreyfus' book appeared in 1972, it received really harsh criticism from the then AI community. Dreyfus actually comments on that criticism in the revised 1992 edition.

I just don't see how Dreyfus critique of AI has been dealt with in modern AI: the critique is aimed at fundamental issues, not at the technical issues.

It is true that the critique written by Dreyfus is based on GOFAI algorithms from the 60s, but it is also true that if you read the book today, you'll find lots of similar situations and a similar way of thinking about the possibilities of AI, as well as the same underlying assumptions.

And as a side note, outdated means that it does not apply anymore, or that is not relevant anymore. Which is different from 'establishing a dialogue' with the text/author, in a way that 'seems' not to be relevant anymore. If you say that Dreyfus' book is outdated just because the 4th edition of Norvig's text only mentions it in a footnote, you are assuming that Norvig and Russell's opinion are definitive. They might be not.

I have authors like Norvig, Russel, LeCun, Minsky and other in the field in high regard. But they are normally not trained in either modern linguistics nor philosophy. Let alone the rest, large amount of researchers in the field. AI research is a complex field, and maybe (in this we could follow Foucault) not even a science. Doing research in an area of study does not turn it into science.

It is precisely philosophy, and even more contemporary philosophy, the discipline that focuses on how we build knowledge, and how we experience the world. Two really important, almost fundamental, topics that directly contribute to how AI is developed as a field of knowledge.

I have the third edition, so I can only speak for it. Being the 3rd edition of the book, I assume that it is the 3rd time the text is revised, so I expect the other two editions (1st and 2nd) to adolesce from the same problem, which I state in the following paragraphs.

The mention to Dreyfus in the 3rd edition of Artificial Intelligence, a Modern Approach, by Stuart Russell and Peter Norvig, is made in 4 different places of the book, referencing four different problems.

The first mention is in page 279, effectively in the bibliographical notes, and it is about something called the 'frame problem'. Dreyfus presents this problem in the 1972 edition of the book, as a problem pertaining 'how to differentiate figure from ground', or 'how to account for what is important and what is not in a specific scenario'. But the solution to the problem that Norvig and Russel cite (Ray Reiter, 1991) is from a paper that _changes the conditions of the problem_, even _change the problem completely_ (by reductionism) to 'how to detect objects that do not change after an action'. They claim the problem solved, but they are actually not addressing Dreyfus criticisms, and misleading the reader to think that the problem is actually solved. The frame problem, by now, is still unsolved (and is one of the most difficult problems to solve).

The second mention is in page 1024, under a section called 'Weak AI: Can machines act intelligently?', and subsection 'The argument from informality'. The section mentions the books What Computers can't do (1972) and What Computers still can't do (1992), as well as Mind over Machine (1986). Unfortunately, this section completely misunderstands the critique of AI that Dreyfus exposes in those books. The whole section is misleading, obfuscating or tergiversing the critique from Dreyfus to fit the purpose of Norvig and Russell (mainly, to show that advances in machine learning and AI can make a solid base for machines that 'act intelligently').

The third mention is in page 1049, and it tries to undermine the first-step fallacy (which is similar to the fallacy of composition). Again, they do it by completely dismissing Dreyfus' critique, not addressing the issue. Then they go on talking about 'rationality' (as explained in chapter 1), but with a trick: only in terms of machines, goal-oriented expectations, computing resources. Dreyfus' critique is about the overall AI enterprise and the search for 'artificial' intelligence, Russell and Norvig discourse in this section first reduce Dreyfus' critique to what they can handle, to their own terms. That is, they evade the issue.

The fourth and final mention, in page 1072, is the bibliographical citation.

Re-reading the non-technical, but more theoretical parts of the book just made me realize how poorly constructed the book is. For example, the definitions given about AI in page 2 are just laughable. Compare with an introductory text on Psychology [0].

[0] https://pressbooks.openeducationalberta.ca/saitintropsycholo...

i recall a hinton lecture where he talks about the mechanism of memory formation in the brain and it's not fully sussed out but it's also not backprop, its some sort of forward prediction and immediate reinforcement loop.

There's also no plausible biological/chemical mechanism to backpropagate.

Again: don't be rude. Not nice. As 'not nice' as my bad formatting.

Philosophy is there precisely to critique and analyze what we build and how we behave in the world. Without it, by the way, there would be no science and no engineering.

Dismissing philosophy in AI is like dismissing philosophy in any other applied, practical or creative field. It is precisely because there is a philosophical investigation on each practical, applied or creative field, that that field can actually make progress.

There's philosophy in biology, which helps biology go further. Philosophy in engineering, which makes engineering go further. In architecture, film, photography, painting, literature, medicine, physics, linguistics, mathematics, etc, etc. In all those fields, there is also a philosophical investigation taking place. Right now.

Maybe you confound practical AI design (you should be talking of 'building AI systems') with AI as a research field. I get that, because lots of people cannot make the distinction (calling yourself 'AI researcher' when you actually tinker a model is not scientific. It might follow a scientific method, but it is engineering research, not scientific research).

Again I don’t think the person you were talking to was doing that.

But one of the reasons that people might do that is because on a forum like hacker news they’ll be attacked as a lunatic. Just look at some of the comments on this article. I saw one where someone said that dualism is so ridiculous that it can’t even be considered philosophy.

Quantum field effects? You don't need these, IMHO, if you look at how highly parallel things seem to work in brains.

Marvin Minsky's theory of a "Society of Mind" describes a (highly) distributed model of the mind. Which BTW, always reminds me of the first Shrek movie, where the donkey jumps up and down, shouting "Take me! Take me!" to Shrek. That's similar to what I observe when I'm undecided but two instances of "sub-processes" (or agents as Minsky calls them) of my mind try to get attention.

Daniel Dennett similarly gives a distributed model of consciousness. Where many parallel "processes" are at work, competing and "observing" each other. And this parallelism is happening with a much, much higher degree than any of our computers parallelism.

They know quite a lot about how neurons work to the extent they can replace bits of brains with artificial retinas or cochleas and interface with devices like neuralink. It's unlikely there is a quantum field effect of the type you mention powering things, although of course atoms and the like obey quantum mechanics in the normal way.

I don't know what point you're making; I'm making the point that it might be harder to discover what makes humans human, than is often suggested. You can't make the same claim about "anything artificial", we know how to dry muddy clay into clay bricks and stack them into a brick wall, and that can be taught from scratch to new people in hours.

You can make a similar argument with a company like ASML where their secret sauce is the organisational ability to fine-tune 100,000 components into a precision Silicon-wafer etching machine. You're far more likely to accidentally stumble upon "how to recreate a mud hut" than "how to recreate ASML". Okay, and...?

i didnt say that at all :-)

Heidegger uses very specific German words to build a very specific vocabulary. This vocabulary then allows him to express very complicated sentiments very quickly and he can use this to express more and more complex structures.

Obviously this requires the reader to first learn the vocabulary and - granted - that is hard and challenging. I have a notebook here , which I consult and modify evertine i dive into Being and Time. But as I said. It keeps on giving. I often try to convey arguments and descriptions to others without falling back into Heideggers jargon, and its sometimes very hard and requires a lot of bloating . So you can argue that it was even necessary for him to invent the vocabulary, because otherwise the book would have been 10x in size

Your comment strikes me as a bit ignorant i must say. You accuse the work of being non-sensical word play but you’ve obviously not invested any time in learning the vocabulary. Because otherwise you wouldnt have made that comment. Id suggest to give it a chance. Its a wonderful piece of work and mind blowing in its own way. That a single mind can think something like that up. I’d argue its on the level of Hegel in terms of system building.

I am not an expert in panpsychism, but for what I know:

1) the idea that everything has a degree of consciousness proportional to its complexity, introduces the problem of compound consciousness. How do they compose, how is each consciousness contributing to the overall, upper-level one? how is experience explained at the different complexity levels?

2) it is impossible to test whether something is conscious or not

3) the theory is more a philosophical framework for dealing with the mind/body problem, but it actually moves the problem forward on the assumption that 'because it is something physical, it has consciousness. Then complex things have complex consciousness'

I think the conversation derailed a bit. I see also a common pattern of jumping through different topics at different levels (from theoretical to concrete and back), and that is confusing.

My original comment was that it seems (and it is actually documented in the books I referenced) that the AI research space builds its claims on assumptions, not on facts, and that those assumptions are flawed. So a nice discussion, to begin with, would consider:

1) why I make the claim that the AI research space builds its claims on assumptions instead of facts, why we could say that there are actually no assumptions but facts, or why the assumptions are correct.

2) instead of strictly and directly dismissing readings on philosophy, I would expect intelligent and curious people to embrace new references. Particularly if those references are highly regarded and a solid contribution during the last 120 years

Regarding point 1), I can barely count a single comment in this thread that tries to engage in the idea of the assumptions (except for some comments that agree with the premise).

Then regarding point 2), I can barely count research papers, books or contributions in the space of AI research that references (either to built upon or dismiss) philosophy that is pertinent to AI, pertinent to philosophy of technique or cognitive linguistics. This is strange. It looks like if the space revived during the 2000s with the invention of neural nets (RL, GAN, etc), and then became isolated from contributions about human intelligence, even though it continually tries to explain intelligence in its own terms.

The reference to What Computers Can't Still do is precisely relevant because it narrates exactly this same discussion (false assumptions, claims built upon assumptions instead of facts, dismissal of evidence from psychology, dismissal of frameworks from philosophy, fallacies about progress), but it was written in 1972. Still, you read the book today, and it is totally relevant.

Now, regarding your comments:

> Do LLM experience objectively or not experience at all? How can you say?

The world cannot be experienced 'objectively'. If they experience the world most probably you won't notice. Given that the only way of interacting with an LLM is through a process initiated solely by a human actor, it would be difficult to assess whether an LLM experiences anything at all.

> I can say that they could think. The thought process implies a measurable product. Yes, there are similar situation with it, it can be hard to say sometimes if we observe a product of a thought or something else. But science has some success with this, like claiming that bees can think and solve problems.

The moment you say 'they could think', that implies an assumption about the actual possibility of thinking as a process that can be modeled and executed by a machine. There is, as far as I know, no current evidence that human beings process information the same way a computer does it, nor that though processes necessarily imply a measurable outcome.

> I'm telling you, psychologists (who specialize on mind research) do not know what consciousness is and they do not have a definition of a subjective experience (well, if we treat your proposed definition as a valid definition, then we should say they have plenty). And my claim still stands: until you heard about new science "Subjectology" you can be sure that no one knows what subjective experience is. Including linguists.

Psychology is a very broad field with lights and shadows through its short history. Here, and in my original comment, I am not talking about linguists in general, but specifically about cognitive linguistics. The contributions made be the field are significant and mostly lacking in AI research (for example, the idea of embodiment, the rebuttal of generative grammars, prototype theory, frame semantics, among others). What you mention as 'subjectology', would be just psychology. Foucault explains more or less clearly why this cannot be a science, and that's just fine (in The Order of Things).

> You shouldn't believe that anything can be settled in the philosophy space. Philosophers can think they have settled things, but until science agreed and started empirical studies, it is just philosophers believing that they settled things.

Well... certainly nothing can be 'settled' (not even in science, btw), but my point is: there is already enough convincing arguments in the field of philosophy so as to say that current LLM systems do not posses agency or experience, and that they do not behave like us.

Again, read the sources, what are you people afraid of? Just read the sources, and then engage in the conversation.

I find it in poor taste, but maybe that’s the point.

Yeah, I think it would be an interesting thought experiment to replace parts of a brain with Neuralink-like devices until you have a fully digital brain, although I doubt it would end up anything like a current LLM.

On top of that, the flow of electrons is seriously slow compared to the speed of the current.

"I am more comfortable speaking to an LLM than a person" is something that should make you reassess yourself, not dismiss the rest of humanity.