The most difficult and time consuming tasks in my job as a software developer is talking to customers and figuring out what they need coupled with getting them to understand what is realistic and what is not. This is a process which takes months if not years, and requires aligning internal goals and tasks with external ones. LLMs can definitely help here, but I feel that the current mode of use where the users have to explicitly manage the tiny - relative to a human brain - context window is an obstacle. I don't know if what we need is a new architecture or just more clever context engineering, but I don't see an LLM actually taking over this type of work as things are now.

LLMs enable a new kind of text search. It looks like reasoning and intelligence, but it is not.

For example, If you are not aware of Internet, you would consider a traditional internet search that comes up with a stackoverflow answer as a machine generating "novel ideas" and answers.

It is a clever marketing trick (touting Erdos solutions) employeed by AI companies.

Wow that sure sounds smart!

The problem, of course, is that generating a one time solution to a problem is a much easier problem space than a many-input task with human product concerns