ChatGPT is already a small agent that receives your message and decides which agent needs to respond. Within those, agents can have sub agents (like when it does research).

When generating images most services will have a small agent that rewrites your request and hands it off to the generative image model.

So from the treasure trove point of view, optimized agents have their place. From companies building pipelines, they also have their place.

> Isn't the entire deal with LLMs that they are trained as megaliths? How can bespoke modelling overcome the treasure trove of knowledge that megaliths can generically bring in, even in bespoke scenarios?

Think of it as a base model (the megalith) which then has the weights adjusted towards a specific use-case (SAP, for example).