Firstly, many of them are still being checked, so hold your horses.

What I’ve found LLMs to be OK at when it comes to novel insight is being given a goal to work towards (“prove or disprove this”), with access to a large body of sources and it can then chase down 1000s of options no human would bother with.

This is why security and maths proofs seem to be an area where it can accelerate things. However I know from my own work in these areas it can still hallucinate and so the output needs careful checking.

Is that novel ideation? Or is it self-organising grid search? It’s more efficient than getting 1000 mathematicians to do the work, but it’s not doing something “new”, it’s building on existing work.

And yes, I know what “attention” means, I was making a pun. And it still stands up: they’re not in some deep thought about this and reflecting on new ideas, they’re just hammering away at everything in the literature.

And it still might be bullshit.