alt Hacker News> So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python.
Tell me you don’t know how llm work without telling me you don’t know how llm work. That’s not how they work!
I'm not actually describing a specific how, I'm describing a goal. The goal is models that are made for specific work, not general work. My model doesn't need to know a single line of python, but if it's a coding model perhaps giving it some python is the best way to make it smarter. Ok. Great - give it python! But first maybe check if giving it C++ is actually better than giving it python. Because for my purposes it doesn't need to know C++ or Python, and I have a fixed parameter budget. Find me the ratio of Swift:C++:Python that gets me the best Swift coding agent in my parameter budget, and then train that.
And then ideally, make it pluggable so I can pick what I want from off the shelf components, but if that's not possible - then just train up as many variants as you can so we can all pick the best variant for our current need.