alt Hacker NewsIt would be interesting to test those LLM-specific features and issues, but I don’t see how it’s a bad study if it does reflect how people actually use LLMs, even if they could use other sources similarly.
The number of people using LLMs must dwarf the number of people using textbooks for any reason.
That would make it a bad study because the stated article title and conclusion is about AI/LLMs but the actual methodology doesn't isolate AI as an independent variable at all. The concept of automation bias is already studied and understood and this just tests groups having to answer "top of head" from their memory against a group given an inaccurate automated system to answer. That doesn't mean that AI doesn't have any of the ill effects people are implying based on the study, it just means this study lacks the rigor to prove or disprove any of those conclusions.