We’re conditioned to interact with language models as chatbots and in that sense strong language understanding (implicit - some sort of world knowledge), is probably necessary for that?

But I’m sure we can have a small model that’s really strong at programming concepts, JavaScript syntax, and that’s about it. You’d interact with it differently, at specific seams in your code base - review a PR, merge two functions together, investigate these logs.

Or maybe I’m just not adequately absorbing the bitter lesson. Idk