alt Hacker NewsFor as rigorous of a Turing test as you present, I believe many (or even most) humans would also fail it.
How many humans seriously have the attention span to have a million "token" conversation with someone else and get every detail perfect without misremembering a single thing?
Replies
nine_k • today at 4:50 PM
But context window exhaustion does not look like mere forgetfulness, but more like loss of general coherence, like getting drunk.
dairem • today at 5:18 PM
Doesn't the Turing test require a human too, to be compared to the AI?
Response quality degrades long before you hit a million tokens.
But sure, let's say it doesn't. If you interact with someone day after day, you'll eventually hit a million tokens. Add some audio or images and you will exhaust the context much much faster.
However, I'll grant you that Turing's original imitation game (text only, human typist, five minutes) is probably pretty close, and that's impressive enough to call intelligence (of a sort). Though modern LLMs tend to manifest obvious dead giveaways like "you're absolutely right!"