alt Hacker News> a flaw in the logic [...] mechanism
Similar to: "Birds fly, my spinning helical device flies, therefore we've started to replicate how birds fly."
> without having to build elements that one expect on a conscious being
One of the elements I expect in a conscious being is that you can't rewrite it by changing the introductory paragraph.
When it comes to LLMs, almost every "mind" we humans perceive is a fictional character in an LLM-generated story-document, one we are either reading or which is being "acted" at us by regular code. Our own instinct for pareidolia and simulating/inferring other minds is very strong, which means we should require really good evidence/logic to counter our instincts.
Even if one believes the LLM has a single "real mind" as an author of every document... what evidence do we have that it is conscious or "self-inserting" itself as one of the characters in the document?
>One of the elements I expect in a conscious being is that you can't rewrite it by changing the introductory paragraph.
If we had enough knowledge of the workings of the human brain, you could alter the perception of every single memory you've ever had. And limited versions of this already happen all the time. Human memory is notoriously unreliable for a reason.
Are you aware of the Recovered Memory Therapy Scandals of the 80s/90s ? Boy did that ruin a lot of lives. You can rewrite a human by changing their 'introductory paragraph'. It's just not as accessible.