She says she can usefully steer LLMs, but how would she know? What knowledge of the research topic, methods and their applicability would she use to judge that, if all she’s doing is writing a prompt asking ChatGPT to write a summary of her research topic, methods and their applicability?

Supposing using LLMs as research tools is fine, the chalk talk is still necessary to test that the researcher has the fundamental skills and knowledge necessary to discriminate usefully between alternative responses, and refine prompts and such. You can’t just get a response from an LLM and then refine it for your specific needs by saying “now refine that reply for my specific needs”. You need to know and understand what those are, and why.

She can’t answer a single thing without opening her laptop.