alt Hacker NewsPeople do use textbooks like that all the time in the experimental setup tested (essentially an open book quiz).
I agree there are important differences in how textbooks and LLMs are used in real life. This study didn't explore that at all. It used a setup that essentially elided the difference between the two.
This is why I think it's a bad study. It didn't measure anything of the essential differences of how people use LLMs.
Replies
nkrisc • last Sunday at 10:45 PM
It would be interesting to test those LLM-specific features and issues, but I don’t see how it’s a bad study if it does reflect how people actually use LLMs, even if they could use other sources similarly.
The number of people using LLMs must dwarf the number of people using textbooks for any reason.
➕ show 1 reply
> People do use textbooks like that all the time in the experimental setup tested (essentially an open book quiz).
What open book quizzes allow you to leave all answers blank with no penalty? An open book quiz is very different from the experimental setup tested here.