This appears to be a 100% AI-generated essay, not the first from ACM and not the last. Your call if you want to spend your day reading and arguing with the article; the author presumably didn't.
I’ve been reading The Moon Is a Harsh Mistress lately and I’ve been shocked at how prescient it is. I’m kind of surprised I haven’t read more about it in the context of today’s AI.
Was recently explaining to my father how the free AI people interact with is completely different than the AI products we're burning cash with. That instead of hitting some critical mass of transistors we hit a critical mass of parameters which we then loop into themselves for reasoning
Other old sci-fi I've been thinking of is The Three Stigmata of Palmer Eldritch. Where claude is chew-z with its alien intelligence & when you start interacting with some code or writing you start noticing that the writer has steel teeth & feel like "damnit, this is palmer eldritch blogging again"
The Culture is what I'd want to happen, the Polity (from Neal Asher) is closer to what I'd expect to happen.
Both are good series if you haven't read one or the other (or either) though I'd say Banks was the better writer, Asher world builds better than anyone I can think of in contemporary science fiction though, Spatterjay is a brilliantly realised genuinely alien feeling world and the Prador are superb "villains" (from a human perspective, totally rational from their own).
I still really enjoy the science fiction of that age. It's fun to read stories set in a universe where computers basically never came into existence. It's really hard for anyone to write stories like that now, so, kind of like analogy to "low-background steel" [1] that has been used recently of the pre-AI world, there's a way in which no more of this sort of story can be produced anymore, or at least, very little. Fortunately there's already more than you can read.
But it is also funny to read about "robots" in a world where computers are still blinking lights and punchcards, if they exist at all. Some of Asimov's work took it to the logical, yet still absurd, conclusion - computers are still blinking lights and punchcards but hyperadvanced and very massive robotic positronic brains are built to run the entire world at once... basically the robots become so advanced that their brains are repurposed as computers. Very amusing in its own way.
Fallout is probably the biggest example of this I can think of. Complete with the robots that are absurdly more capable than the computer hardware seems to be, and the Yes Man who at least aspires to run his part of the world.
I have been re-reading Children of the Mind (sequel to Ender's Game) and I had forgotten about Jane, who is an AI that talks to Ender through his earpiece, seeing and hearing everything he does. Jane proactively searches things for him, manages his finances and offers advice, as well as friendly banter. In the novel, the AI is treated like an AGI that treats only him this way, but I realized how very close we are to that reality right now, for everyone.
All of the use cases and integrations for AI seem to be leading toward a future exactly like this. It was interesting to see how reliant he became on his AI (couldn't use a computer, didn't know his own password) - it is barely scifi at this point, and terrifying to see how close that future is, how useful it would be, and how dangerous.
The Internet has globalized/flattened elements of culture very quickly, and AI will only accelerate that. And knowing how vulnerable humans are to suggestion/advertisement/propaganda, it seems like a perfect recipe for disaster. From an evolutionary standpoint, monoculture provides certain short-term advantages but centralizes weaknesses. If nothing else, I think we are in for a bumpy ride, whether or not we continue to boil the ocean.
“Much of today’s discourse on AI consciousness rests not on empirical evidence but on functionalism …”
In my view there can never be an empirical test for consciousness, because functionalism is wrong. If functionalism was right then there would be one. There’s no third party test for a fundamentally first person phenomenon.
this would require us to still be baffled by consciousness after knowing everything about neurophysiology that might be relevant - until then, the keys to unlocking its secrets could still be hidden in our wetware behind a veil of ignorance
It will still be baffling that there is consciousness.
Maybe if you start messing with parts of my brain so I can become familiar with the third person correlates of consciousness (which in my usage refers entirely to the first person part) then I could learn something, but I could never learn why or how the first person exists. It just does.
If we perfectly reduce the brain to some scientific model, and I don’t doubt we will, we’ll be able to predict what I will say, but not understand why I’m observing it being said.
Can you learn anything about 'why 8?' or 'how 7?'.
If you pose an incoherent question, the answer is 'that's not a valid question', rather than 'oh wow, how baffling'. The baffling is coming from forming what seem to be valid questions that aren't.
Another example of it is that famous who cuts the barber's hair (Russell's paradox):
In a village, the barber shaves all and only those men who do not shave themselves.
Who shaves the barber?
Here's an LLM explaining it:
the philosophical/logical lesson is: sometimes a description that sounds grammatically fine does not actually describe a possible thing. The contradiction is in the premise, not in our failure to find the right barber.
---
For clarity, consciousness is like free will - if you invent a thing that doesn't exist and proceed to ask questions about it, the problem is you having invented a thing, not that you are then having difficulty answering questions about it.
I recall reading that some thought that simple electrical signals could not fully explain our nervous system and there may be some quantum effects in the mix.
This is the question of qualia - and it's surprisingly intractable. Realize that in this context, I can't be sure that you are conscious, either. Arguments here will crash into solipsism in the end: you guys can never prove you're not just one of my dreams.
I think that was the ultimate point of the Turing test. We can't know for certain - so the best we can do is just ask.
Any argument that would lead us to discard 99.9999999% of the phenomenon under debate, no matter how reasonable, is certainly not useful.
That's why I hold with the philosophical stance of "consciousness is the thing that makes humans talk about consciousness."
"What if reality doesn't exist?" The point of thought is to analyze reality. If your thinking leads you to discard the subject of the matter, well, feel free to do that but I don't see why anyone else should take it seriously.
> 100% of this text is AI
https://www.pangram.com/history/f8a0cf46-1d3d-4fea-a083-d88c...
Other old sci-fi I've been thinking of is The Three Stigmata of Palmer Eldritch. Where claude is chew-z with its alien intelligence & when you start interacting with some code or writing you start noticing that the writer has steel teeth & feel like "damnit, this is palmer eldritch blogging again"
Both are good series if you haven't read one or the other (or either) though I'd say Banks was the better writer, Asher world builds better than anyone I can think of in contemporary science fiction though, Spatterjay is a brilliantly realised genuinely alien feeling world and the Prador are superb "villains" (from a human perspective, totally rational from their own).
The Player of Games is an easy read if anyone wants to try him for the first time.
It's one of the reason that Musk referencing The Culture makes me want to smash my head on a desk (or someone's at least).
On a personal level I suspect Banks would have deeply disliked Musk (based on his political leanings during his life time and his works in aggregate).
But it is also funny to read about "robots" in a world where computers are still blinking lights and punchcards, if they exist at all. Some of Asimov's work took it to the logical, yet still absurd, conclusion - computers are still blinking lights and punchcards but hyperadvanced and very massive robotic positronic brains are built to run the entire world at once... basically the robots become so advanced that their brains are repurposed as computers. Very amusing in its own way.
Fallout is probably the biggest example of this I can think of. Complete with the robots that are absurdly more capable than the computer hardware seems to be, and the Yes Man who at least aspires to run his part of the world.
[1]: https://en.wikipedia.org/wiki/Low-background_steel
All of the use cases and integrations for AI seem to be leading toward a future exactly like this. It was interesting to see how reliant he became on his AI (couldn't use a computer, didn't know his own password) - it is barely scifi at this point, and terrifying to see how close that future is, how useful it would be, and how dangerous.
The Internet has globalized/flattened elements of culture very quickly, and AI will only accelerate that. And knowing how vulnerable humans are to suggestion/advertisement/propaganda, it seems like a perfect recipe for disaster. From an evolutionary standpoint, monoculture provides certain short-term advantages but centralizes weaknesses. If nothing else, I think we are in for a bumpy ride, whether or not we continue to boil the ocean.
In my view there can never be an empirical test for consciousness, because functionalism is wrong. If functionalism was right then there would be one. There’s no third party test for a fundamentally first person phenomenon.
Maybe if you start messing with parts of my brain so I can become familiar with the third person correlates of consciousness (which in my usage refers entirely to the first person part) then I could learn something, but I could never learn why or how the first person exists. It just does.
If we perfectly reduce the brain to some scientific model, and I don’t doubt we will, we’ll be able to predict what I will say, but not understand why I’m observing it being said.
If you pose an incoherent question, the answer is 'that's not a valid question', rather than 'oh wow, how baffling'. The baffling is coming from forming what seem to be valid questions that aren't.
Another example of it is that famous who cuts the barber's hair (Russell's paradox):
In a village, the barber shaves all and only those men who do not shave themselves. Who shaves the barber?
Here's an LLM explaining it:
the philosophical/logical lesson is: sometimes a description that sounds grammatically fine does not actually describe a possible thing. The contradiction is in the premise, not in our failure to find the right barber.
---
For clarity, consciousness is like free will - if you invent a thing that doesn't exist and proceed to ask questions about it, the problem is you having invented a thing, not that you are then having difficulty answering questions about it.
I think that was the ultimate point of the Turing test. We can't know for certain - so the best we can do is just ask.
Agreed. But that also means I have no empirical evidence that YOU are conscious either.
That's why I hold with the philosophical stance of "consciousness is the thing that makes humans talk about consciousness."
"What if reality doesn't exist?" The point of thought is to analyze reality. If your thinking leads you to discard the subject of the matter, well, feel free to do that but I don't see why anyone else should take it seriously.