alt Hacker NewsI would only start to entertain this when we had AI running for reasonably long period say a year with same full context. That is in same run over a year it had collected information. In singular session.
No point even considering it if it can't run for extended periods. And actually keeping learning during the duration. As surely person should stay alive for decades if allowed.
But you could still prompt it to act like any person, including fictional and deceased ones, which proves it has no real concept of self. It's just a language model for humans to generate whatever text meets the task. You could have it run for a year as different perspectives in GRR Martin's ASOIAF to finish his books for him. And then what, award it with Jon Snow's personhood?
An AI agent would need some kind of stable personality of it's own. Some sense of self outside whatever the hell people task it with for this to be meaningful. Data from Star Trek is a good fictional benchmark for that. Or Voyager's Holographic doctor.