alt Hacker NewsI just can't understand the certainty that people have about the limitations of models.
The diagrams in this article are just outright wrong.
Models are not 1.prompt->2.forward propagation->3.response->4.end.
They are 1.prompt->2.forward propagation->3.partial response->if not done goto 2.-> end.
And if you can't see how that adds a world of complexity I'm not sure what else to say. AI may not have intent or motivation, but the ability to show that is currently well beyond our means.
you are not wrong regarding your example. I think personally LLMs offer a new level of abstraction to do work. it doesnt have intent, i think its even silly to think about it. You specify your intent and it has agency to pursue that. in programming field its really obvious, as its not unlike specifying intent through code and having other programs pursue that intent (perhaps more determenistically) through the agency contained in them.
all the 'bad stuff' they do like hacking stuff or taking shortcuts is humans not understanding how deep the rabbithole goes of specifying intent clearly in light of giving something capabilities and a reward to acheive a goal without specifying _exactly _how to use the capabilities (which is what a classic program would be).